TCM Target Install and Operation Guide
- Install Linux OS. Example here is RHEL 6.1
- Make sure the followings are installed
fcoe - select "Base Systems", then "fcoe storage client"
yum
git
gcc
python
- After OS is up and running:
- Disable Fire Wall
chkconfig iptables off
chkconfig ip6tables off
- Disable SElinux
edit "/etc/selinux/config"
SELINUX=disabled
- Bring up network interface
edit "/etc/sysconfig/network-scripts/ifcfg-ethX"
ONBOOT=yes
- Run the following steps to bring up DCB and FCoE initiator (Examples below using interface eth2)
- Bring up dcb
dcbtool sc eth2 dcb on
dcbtool sc eth2 pfc e:1 a:1 w:1
dcbtool sc eth2 app:0 e:1
- Copy cfg-ethx file
cp /etc/fcoe/cfg-ethx /etc/fcoe/cfg-eth2
- Run fip discovery and create fcoe interface
fipvlan -a
fipvlan -c eth2
- confirm ethx.<vlan ID>-fcoe (e.g., eth2.152-fcoe) is created
ifconfig | grep eth2
- Create fcoe interface
fcoeadm -c eth2.152-fcoe
- Confirm fcoe interface is up and running:
fcoeadm -i
- Download Upstream kernel source
- Download fcoe upstream kernel "fcoe-next" from open-fcoe.org
git clone git://open-fcoe.org/fcoe/fcoe-next.git
OR get a tar ball from
http://www.open-fcoe.org/git/?p=fcoe/fcoe-next.git;a=tree
- click "Snapshot" in the upper left corner under "summary" to download
- Install fcoe-next kernel
- copy the upstream kernel source directory in a proper location, in example below, we use:
/usr/src/kernel/
- cd /usr/src/kernel/<upstream kernel source PWD>, example:
cd /usr/src/kernel/fcoe-next-xyz/
- copy ".config" file from original kernel source directory to upstream kernel source directory:
cp ../2.6.32-131.0.15.e16.x86_84/.config .
- run the folowing to config and build kernel
make menuconfig
- select/deselct the following:
Device Drivers -> SCSI device support -> [*] SCSI low-level drivers
[M] LibFC module
[M] LibFCoE module
[M] FCoE module
deselect [ ]iSCSI Initiator over TCP/IP
deselect [ ]Cisco FNIC Driver
Device Drivers -> [M] Generic Target Core Mod (TCM) and ConfigFS Infrastructure
[M] TCM/IBLOBK Subsystem Plugin for Linux/BLOCK
[M] TCM/FILEIO Subsystem Plugin for Linux/VFS
[M] TCM/pSCSI Subsystem Plugin for Linux/SCSI
[M] TCM/STGT Subsystem Plugin
[M] TCM Virtual SAS target and Linux/SCSI LDD fabric loopback module
[*] TCM loopback fabric module CDB debug code
[M] TCM_FC fabric Plugin
Device Drivers -> [*] Network device support -> [*] Ethernet (10000 Mbit)
[M] Intel(R) 10GbE PCI Express adapter support
[*] Direct Cache Access (DCA) support
[*] Data Center Bridging (DCB) support
deselect [ ]Wireless LAN
File systems -> Psuedo filesystems
[M] Userspace-driven configuraton filesystem
NOTE: "Userspace-driven configuraton filesystem" can be compiled in as a module or built-in into kernel. After complining the kernel make sure configfs is loaded as a module and mounted. If not, please use following 2 commands:
modprobe configfs
mount -t configfs none /sys/kernel/config/
- deselect the followings:
[*] Networking support -> Networking options
Asynchronous Transfer Mode (ATM)
[*] Networking support
WiMAX Wireless Broadband support
RF switch subsystem support
Device Drivers
ISDN support
InfiniBand support
Xen driver support
Virtualization
- compile and build the kernel
make && make modules_install && make install
- Edit the boot menu to boot from the new kernel
- Reboot the system and boot to the new kernel
- NOTE: If you are not use RHEL 6.1, refer to "QUICKSTART: from the following link for anything new in user space tool installation procedures
http://www.open-fcoe.org/git/?p=fcoe/fcoe-utils.git;a=blob;f=QUICKSTART;h=69741db7b62d6ab16030e7c7318e5f8b66b216e8;hb=HEAD
- Installing TCM target user mode tools and other necessary software packages
- Installing other software packages
cd /usr/local/lib
ln –s /usr/lib/python2.6/ python2.6
yum install –y net-snmp
yum install –y net-snmp-devel
- Download and install user mode tools
git://risingtidesystems.com/lio-utils.git
cd lio-utils
make install
- TCM LIO target service commands
/etc/init.d/target start
/etc/init.d/target stop
Or
service target start
service target stop
- Example of create a new target. The following example:
- creates a 100 MB file, named file0 in directory /home/tgt/
- use this file as LUN 0 and mapped to the target fileio_0
- expose this target to the designated initiator
mkdir /home/tgt
tcm_node --fileio fileio_0/file0 /home/tgt/file0 100000000
tcm_node --wwn fileio_0/file0
lio_node --addlun fc-test 1 0 lun0 fileio_0/file0
lio_node --listendpoints
lio_node --demomode fc-test 1
lio_node --disableauth fc-test 1
modprobe fcoe
modprobe tcm_fc
mkdir -p /sys/kernel/config/target/fc/20:00:00:1b:21:55:1c:da/tpgt_1/lun/lun_0
cd /sys/kernel/config/target/fc/20\:00\:00\:1b\:21\:55\:1c\:da/tpgt_1/lun/lun_0/
ln -s /sys/kernel/config/target/core/fileio_0/file0 tcm_fc_port
cd ../../ ; mkdir -p acls/20:00:00:1b:21:37:b8:df/lun_0
cd acls/20\:00\:00\:1b\:21\:37\:b8\:df/lun_0
ln -s /sys/kernel/config/target/fc/20\:00\:00\:1b\:21\:55\:1c\:da/tpgt_1/lun/lun_$i/ lun_$i
Here, the WWPN
20:00:00:1b:21:55:1c:da is for the target
20:00:00:1b:21:37:b8:df is for the initiator
NOTE:
- You can also use block device, such as actual hard disks or ramdisk as the LUNs, the tcm_node command should be adjusted accordingly. EXample:
tcm_node --iblock iblock_0/block0 /dev/sdc
tcm_node --wwn iblock_0/block0
lio_node --addlun fc-test 1 0 lun0 iblock_0/block0
- You can create a script to add multiple LUNs to the target, example:
for i in 1 2 3 4; do
tcm_node --fileio fileio_0/file$i /home/tgt/file$i 100000000
tcm_node --wwn fileio_0/file$i
lio_node --addlun fc-test 1 $i lun0 fileio_0/file$i
done
- When running the above commands, a number of warning messages may occur, such as
"Unable to create iSCSI target logical unit configFS directory"
"OSError: No such file or directory: /sys/kernel/config/target/iscsi"
"Unable to disable ISCSI authentication on iSCSI target portal greoup"
This is because TCM is used by multple targets, include iSCSI and FCoE targets; and some of the codes are not cleaned up properly at this time.
- Fabric configuration
- There are two methods for fabric configuration: zoning and direct virtual ports connection:
- Fabric zone:
- Perform typical zoning; configure the target and the initiator in the same zone.
- After zoing, the initiator should be able to access the target (LUNs).
- You may also need to reset your initiator to have the target/LUNs showing up.
- Virtual N-port to virtual N-port (back-to-back) connection:
- run the following commands to create vn-to-vn interface in both initiator and target:
echo eth2 > /sys/module/libfcoe/parameters/create_vn2vn
assuming using interface eth2
- to delete the fcoe interface, run the following commands:
echo eth2 > /sys/module/libfcoe/parameters/destroy
- Reference:
- Type "tcm_node -h" to show all tcm_node command parameters
- Type "lio_node -h" to show all lio_node command parameters