Fixing Windows Telnet

Breaking with tradition and posting something about Windows again….

One thing that I have always hated about using a Windows PC is when I pull up a command prompt to telnet to a mail server or something to test.  If you’ve ever tried this you know what I’m talking about – you can’t see what you are typing!  It’s not that bad, if you type perfectly all the time, but I do not.  Well, I was looking something up on MSDN today and happened to stumble upon this gem:  “set localecho”.  You have to specify it within the telnet prompt – you can’t specify it in one line on the command prompt, but at least it’s something.

From the command prompt, type just telnet, instead of the full line:

> telnet

Type in set localecho and then open the server port that you want:

telnet> set localecho
telnet> open mailserver.domainname.com 25

Delete multiple files that contain spaces based on age

I had a problem on a FTP server running Solaris 9 that had tons of old files and many of them had spaces in the names. How do you automate removing the old files (the real concern being the spaces in the filenames)? Most solutions recommend using -print0 in find or xargs -0, neither of which are options in Solaris 9.

This command removes any files older then 2 years old in the current directory:

find . -type f -mtime +730 | while read filename; do rm -f “$filename”; done

Adding a Drive to a Veritas Disk Group

Last time I showed you how to add a new drive to make a new disk group. Well, what if you already have a disk group, with just one drive on it and you want to mirror that data across another drive? The process is easy and here’s how:

Let’s start off with a simple disk group:

# vxprint
Disk group: tempdg-tmp

TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0
dg tempdg-tmp tempdg-tmp – – – – – –

dm tempdg-tmp02 emcpower1s2 – 733975296 – – – –

v templv-tmp fsgen ENABLED 733975296 – ACTIVE – –
pl templv-tmp-02 templv-tmp ENABLED 733975296 – ACTIVE – –
sd tempdg-tmp02-01 templv-tmp-02 ENABLED 733975296 0 – – –

This shows the disk group’s name is tempdg-tmp, the original disk in the group is called tempdg-tmp02, the logical volume is called templv-tmp, which is made up of one plex called templv-tmp-02, which has one subdisk called tempdg-tmp02-01.

So, let’s add an additional disk tempdg-tmp01 and use it to mirror the data on tempdg-tmp02. If we look first at how the device is setup for the original drive we see that it is using cdsdisk format:

# vxdisk -o alldgs list
DEVICE TYPE DISK GROUP STATUS
emcpower1s2 auto:cdsdisk tempdg-tmp02 tempdg-tmp online

So let’s make our new device cdsdisk formatted as well:

# vxdisk -o alldgs list
DEVICE TYPE DISK GROUP STATUS
emcpower6s2 auto:none – – online invalid

# /etc/vx/bin/vxdisksetup -i emcpower6 format=cdsdisk

# vxdisk -o alldgs list
DEVICE TYPE DISK GROUP STATUS
emcpower6s2 auto:cdsdisk – – online

Now to add the new disk to the disk group. Note that since the original drive was called 02, the one we are adding is 01:

# vxdg -g tempdg-tmp adddisk tempdg-tmp01=emcpower6

Now we can see the new drive in the disk group (and that it isn’t currently doing anything):

# vxprint
Disk group: tempdg-tmp

TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0
dg tempdg-tmp tempdg-tmp – – – – – –

dm tempdg-tmp01 emcpower6s2 – 733975296 – – – –
dm tempdg-tmp02 emcpower1s2 – 733975296 – – – –

v templv-tmp fsgen ENABLED 733975296 – ACTIVE – –
pl templv-tmp-02 templv-tmp ENABLED 733975296 – ACTIVE – –
sd tempdg-tmp02-01 templv-tmp-02 ENABLED 733975296 0 – – –

So let’s tell the disk group to set tempdg-tmp01 up as a mirror of tempdg-tmp02:

# vxassist -g tempdg-tmp mirror templv-tmp tempdg-tmp01 &

I ran it in the background because it can take a long time depending on how big it is (about 2 hours for 350 GB). To watch the mirroring process, use vxtask. (You can also use vxprint to see what it is doing.)

# vxtask list
TASKID PTID TYPE/STATE PCT PROGRESS
177 ATCOPY/R 04.89% 0/733975296/35921920 PLXATT templv-tmp templv-tmp-01 tempdg-tmp

Once it completes, you will see your new drive setup as a mirror:

# vxprint
Disk group: tempdg-tmp

TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0
dg tempdg-tmp tempdg-tmp – – – – – –

dm tempdg-tmp01 emcpower6s2 – 733975296 – – – –
dm tempdg-tmp02 emcpower1s2 – 733975296 – – – –

v templv-tmp fsgen ENABLED 733975296 – ACTIVE – –
pl templv-tmp-02 templv-tmp ENABLED 733975296 – ACTIVE – –
sd tempdg-tmp02-01 templv-tmp-02 ENABLED 733975296 0 – – –
pl templv-tmp-01 templv-tmp ENABLED 733975296 – ACTIVE – –
sd tempdg-tmp01-01 templv-tmp-01 ENABLED 733975296 0 – – –

Create new Veritas Disk Group

Earlier I posted about how to add a new LUN to a Solaris 9 server. Today I’ll go through the process of adding that disk (emcpower6) to a new disk group and making it usable with a file system. This server has EMC Powerpath installed as well as the Veritas Storage Foundation running on it.

First, setup the disk inside Veritas, in this example I’ll use the sliced format (emcpower6 is the Veritas Device name – not the PowerPath pseudo name – use vxdisk list to see the name):

# /etc/vx/bin/vxdisksetup -i emcpower6 format=sliced

Initiate the disk group by adding the disk to a new group (tempdg-tmp). The device name is emcpower6 and the name of the disk inside the disk group will be called tempdg-tmp01:

# vxdg init tempdg-tmp cds=off tempdg-tmp01=emcpower6

See how big we can make the disk group:

# vxassist -g tempdg-tmp maxsize
Maximum volume size: 733952000 (358375Mb)

Setup the disk group to the maximum size (templv-tmp is the logical volume name):

# vxassist -g tempdg-tmp -b make templv-tmp 733952000 tempdg-tmp01 layout=nostripe

At this point you can take a look at what you created with vxprint:

# vxprint
Disk group: tempdg-tmp

TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0
dg tempdg-tmp tempdg-tmp – – – – – –

dm tempdg-tmp01 emcpower6s2 – 733952000 – – – –

v templv-tmp fsgen ENABLED 733952000 – ACTIVE – –
pl templv-tmp-01 templv-tmp ENABLED 733952000 – ACTIVE – –
sd tempdg-tmp01-01 templv-tmp-01 ENABLED 733952000 0 – – –

Create a file system on the logical volume, using VXFS:

# /opt/VRTS/bin/mkfs -F vxfs /dev/vx/rdsk/tempdg-tmp/templv-tmp
version 6 layout
733952000 sectors, 366976000 blocks of size 1024, log size 65536 blocks
largefiles supported

Create a mount point, mount it, see it mounted:

# mkdir /u2

# mount -F vxfs /dev/vx/dsk/tempdg-tmp/templv-tmp /u2

# df -h /u2
Filesystem size used avail capacity Mounted on
/dev/vx/dsk/tempdg-tmp/templv-tmp 350G 153M 328G 1% /u2

Adding a new LUN on Solaris 9 with VXFS and PowerPath

There’s not a lot of documentation out there on working with LUNs on servers that use both Veritas Storage Foundation and EMC PowerPath. I have spent the last few weeks perfecting the process of Dynamic LUN Expansion and will be adding more posts about that process in the near future. Most recently I have been working on Solaris 9 servers connected to an EMC Clariion CX500 SAN that have LUNs mounted using the VXFS (Veritas) file system (version 4.1,REV=4.1B18_sol_GA_s10b74L2a) using EMC PowerPath (version 4.5.0_b169). The servers have Qlogic HBAs so I use their utilities to scan the Fibre Channel paths. If you have a different HBA, use that vendor’s tools.

This setup, although not unusual, does make it more complicated. When a new LUN is connected to the host, by default, Veritas will see it first and claim the drive. This is a bad thing, because we want PowerPath to find the drive, setup the pseudo drive and encapsulate the multipathing.

Previous administrators with this setup were unable to get PowerPath to recognize the drive before Veritas without rebooting the server. They would connect the LUN and reboot the server. If Veritas detected the drive first, they would remove the disk from Veritas and reboot until PowerPath claimed the drive. This, of course, to me was unacceptable, so I have decided to document the procedure I follow for properly connecting a new LUN to a host with this configuration. (Note: If you have already got yourself in this situation, you will still need to reboot to cleanup Veritas’ hold on the drive. The alternative to rebooting would be to allocate another LUN for the host and worry about cleaning up the previous one later.)

So how do you get Veritas to not claim the drive when you connect the LUN to the host? Simple, turn off the service that discovers the drive! If you dig into the init scripts for Veritas (/etc/init.d/vxvm-startup2), you’ll see starting on line 193 the following lines:

# Start Event Source Daemon, to enable dynamic device discovery
# Empty the lock file before starting the daemon
> /etc/vx/.vxesd.lock

vxddladm start eventsource

This dynamic device discovery service must be turned off by executing the following command:

# vxddladm stop eventsource

Of course, if you don’t want dynamic discovery service running, you should also comment out line 197 so that it is never started in the first place. I have found that it is necessary to run the stop command multiple times to make sure it is off. Even if I have stopped it on a particular server before, I run it again before connecting any LUNs, just to make sure.

So here are the steps with sample output if any was generated:

Turn off Veritas dynamic discovery (IMPORTANT!!):

# vxddladm stop eventsource

Rescan your fibre connections, optionally verify that you can see the new LUN:

# /opt/QLogic_Corporation/SANsurferCLI/scli -do all rescan
Driver rescan completed on HBA instance 0.
Driver rescan completed on HBA instance 1.

Tell Solaris you’ve made some changes and let it make the proper devices:

# devfsadm -v -C

Configure PowerPath to work with the new LUN:

# /etc/powercf -q
# /etc/powermt config
# /etc/powermt save

Check that your new LUN is properly configured in PowerPath:

# /etc/powermt display dev=all
— cut —
Pseudo name=emcpower6a
CLARiiON ID=APM00012345678 [Test Storage Group]
Logical device ID=60060160A42611000A2B03D123456789 [LUN – TEMP LUN]
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP A, current=SP A
===============================================
—————- Host ————— – Stor – — I/O Path – — Stats —
### HW Path I/O Paths Interf. Mode State Q-IOs Errors
===============================================
1280 pci@9/fibre-channel@2 c4t1d6s0 SP B0 active alive 0 0
1280 pci@9/fibre-channel@2 c4t2d6s0 SP A0 active alive 0 0
1281 pci@9/fibre-channel@2 c5t1d6s0 SP B1 active alive 0 0
1281 pci@9/fibre-channel@2 c5t2d6s0 SP A1 active alive 0 0
— cut —

Now we need to create a label on the drive, use the PowerPath pseudo name:

# format
Searching for disks…done

c4t1d6: configured with capacity of 349.99GB
c4t2d6: configured with capacity of 349.99GB
c5t1d6: configured with capacity of 349.99GB
c5t2d6: configured with capacity of 349.99GB
emcpower6a: configured with capacity of 349.99GB

AVAILABLE DISK SELECTIONS:
0. c1t0d0
/pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w210000008710a5b2,0
— cut —
40. emcpower6a
/pseudo/emcp@6
Specify disk (enter its number): 40
selecting emcpower6a
[disk formatted]
Disk not labeled. Label it now? yes
format> q

Get rid of the default partitions created by the label command:

# fmthard -d 0:00:00:0:0 /dev/rdsk/emcpower6a
# fmthard -d 1:00:00:0:0 /dev/rdsk/emcpower6a
# fmthard -d 6:00:00:0:0 /dev/rdsk/emcpower6a

Verify that everything looks the way you want it:

# format
Searching for disks…done

AVAILABLE DISK SELECTIONS:
0. c1t0d0
/pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w210000008710a5b2,0
— cut —
40. emcpower6a
/pseudo/emcp@6
Specify disk (enter its number): 40
selecting emcpower6a
[disk formatted]
format> verify

Primary label contents:

Volume name =
ascii name =
pcyl = 57344
ncyl = 57342
acyl = 2
nhead = 256
nsect = 50
Part Tag Flag Cylinders Size Blocks
0 unassigned wm 0 0 (0/0/0) 0
1 unassigned wm 0 0 (0/0/0) 0
2 backup wu 0 – 57341 349.99GB (57342/0/0) 733977600
3 unassigned wm 0 0 (0/0/0) 0
4 unassigned wm 0 0 (0/0/0) 0
5 unassigned wm 0 0 (0/0/0) 0
6 unassigned wm 0 0 (0/0/0) 0
7 unassigned wm 0 0 (0/0/0) 0

format> q

If you’ve done everything right, Veritas should not see your new drive yet. Verify:

# vxdisk -o alldgs list
DEVICE TYPE DISK GROUP STATUS
emcpower0s2 auto:sliced devdg-erp02 devdg-erp online
emcpower1s2 auto:cdsdisk testdg-erp02 testdg-erp online
emcpower2s2 auto:sliced devdg-dw02 devdg-dw online
emcpower3s2 auto:sliced devdg-tle01 devdg-tle online
emcpower4s2 auto:sliced dbadg-erp01 dbadg-erp online

Now that everything is setup right, go ahead and tell Veritas to look for new drives. Important Note: If it is not setup right at this point, Veritas will claim the raw device and it’s a pain to cleanup!

# vxdctl enable

Now, you will see the pseudo drive in the list. If it’s a raw device name, you messed up:

# vxdisk -o alldgs list
DEVICE TYPE DISK GROUP STATUS
emcpower0s2 auto:sliced devdg-erp02 devdg-erp online
emcpower1s2 auto:cdsdisk testdg-erp02 testdg-erp online
emcpower2s2 auto:sliced devdg-dw02 devdg-dw online
emcpower3s2 auto:sliced devdg-tle01 devdg-tle online
emcpower4s2 auto:sliced dbadg-erp01 dbadg-erp online
emcpower6s2 auto:none – – online invalid

You should now see new devices in /dev/vx/dmp and /dev/vx/rdmp. What’s next? Well, that’s entirely up to you. Now that you’ve got the right drive in Veritas, you can set it up anyway you want – add it to an existing disk group, mirror a disk group for some added redundancy, create a new disk group, etc. In upcoming posts, I’ll detail how to create a new disk group with this LUN and then start digging into dynamically resizing file systems while everything is still online and available to the users.