User Tools

Site Tools


TCM FCoE Target Guide

TCM Target Install and Operation Guide

- Install Linux OS.  Example here is RHEL 6.1
 - Make sure the followings are installed
     fcoe - select "Base Systems", then "fcoe storage client"

- After OS is up and running:
 - Disable Fire Wall
  chkconfig iptables off
  chkconfig ip6tables off

 - Disable SElinux
  edit "/etc/selinux/config"

 - Bring up network interface
  edit "/etc/sysconfig/network-scripts/ifcfg-ethX"

- Run the following steps to bring up DCB and FCoE initiator (Examples below using interface eth2)
 - Bring up dcb
  dcbtool sc eth2 dcb on
  dcbtool sc eth2 pfc e:1 a:1 w:1
  dcbtool sc eth2 app:0 e:1
 - Copy cfg-ethx file
  cp /etc/fcoe/cfg-ethx /etc/fcoe/cfg-eth2

 - Run fip discovery and create fcoe interface
  fipvlan -a
  fipvlan -c eth2
 - confirm ethx.<vlan ID>-fcoe (e.g., eth2.152-fcoe) is created
  ifconfig | grep eth2
 - Create fcoe interface
  fcoeadm -c eth2.152-fcoe

 - Confirm fcoe interface is up and running:
  fcoeadm -i

- Download Upstream kernel source
 - Download fcoe upstream kernel "fcoe-next" from

  git clone git://

 OR get a tar ball from;a=tree
   - click "Snapshot" in the upper left corner under "summary" to download

- Install fcoe-next kernel

 - copy the upstream kernel source directory in a proper location, in example below, we use:

 - cd /usr/src/kernel/<upstream kernel source PWD>, example:
  cd /usr/src/kernel/fcoe-next-xyz/

 - copy ".config" file from original kernel source directory to upstream kernel source directory:
  cp ../2.6.32-131.0.15.e16.x86_84/.config .
 - run the folowing to config and build kernel
  make menuconfig

 - select/deselct the following:
Device Drivers -> SCSI device support -> [*] SCSI low-level drivers
 [M] LibFC module
 [M] LibFCoE module
 [M] FCoE module

 deselect [ ]iSCSI Initiator over TCP/IP
 deselect [ ]Cisco FNIC Driver

Device Drivers -> [M] Generic Target Core Mod (TCM) and ConfigFS Infrastructure
 [M] TCM/IBLOBK Subsystem Plugin for Linux/BLOCK
 [M] TCM/FILEIO Subsystem Plugin for Linux/VFS
 [M] TCM/pSCSI Subsystem Plugin for Linux/SCSI
 [M] TCM/STGT Subsystem Plugin
 [M] TCM Virtual SAS target and Linux/SCSI LDD fabric loopback module
 [*] TCM loopback fabric module CDB debug code
 [M] TCM_FC fabric Plugin

Device Drivers -> [*] Network device support -> [*] Ethernet (10000 Mbit)
 [M] Intel(R) 10GbE PCI Express adapter support
 [*] Direct Cache Access (DCA) support
 [*] Data Center Bridging (DCB) support

 deselect [ ]Wireless LAN

File systems -> Psuedo filesystems
 [M] Userspace-driven configuraton filesystem

NOTE: "Userspace-driven configuraton filesystem" can be compiled in as a module or built-in into kernel. After complining the kernel make sure configfs is loaded as a module and mounted. If not, please use following 2 commands:

    modprobe configfs

    mount -t configfs none /sys/kernel/config/

 - deselect the followings:
[*] Networking support -> Networking options
 Asynchronous Transfer Mode (ATM)
[*] Networking support 
 WiMAX Wireless Broadband support
 RF switch subsystem support
Device Drivers 
 ISDN support
 InfiniBand support
 Xen driver support

 - compile and build the kernel
  make && make modules_install && make install

 - Edit the boot menu to boot from the new kernel

 - Reboot the system and boot to the new kernel

- NOTE: If you are not use RHEL 6.1, refer to "QUICKSTART: from the following link for anything new in user space tool installation procedures;a=blob;f=QUICKSTART;h=69741db7b62d6ab16030e7c7318e5f8b66b216e8;hb=HEAD

- Installing TCM target user mode tools and other necessary software packages

 - Installing other software packages
  cd /usr/local/lib
  ln –s /usr/lib/python2.6/ python2.6

  yum install –y net-snmp
  yum install –y net-snmp-devel

 - Download and install user mode tools

  cd lio-utils
  make install

- TCM LIO target service commands
 /etc/init.d/target start
 /etc/init.d/target stop
 service target start
 service target stop

- Example of create a new target.  The following example:
 - creates a 100 MB file, named file0 in directory /home/tgt/
 - use this file as LUN 0 and mapped to the target fileio_0
 - expose this target to the designated initiator 

 mkdir /home/tgt
 tcm_node --fileio fileio_0/file0 /home/tgt/file0 100000000
 tcm_node --wwn fileio_0/file0
 lio_node --addlun fc-test 1 0 lun0 fileio_0/file0

 lio_node --listendpoints
 lio_node --demomode fc-test 1
 lio_node --disableauth fc-test 1

 modprobe fcoe
 modprobe tcm_fc

 mkdir -p /sys/kernel/config/target/fc/20:00:00:1b:21:55:1c:da/tpgt_1/lun/lun_0

 cd /sys/kernel/config/target/fc/20\:00\:00\:1b\:21\:55\:1c\:da/tpgt_1/lun/lun_0/

 ln -s /sys/kernel/config/target/core/fileio_0/file0 tcm_fc_port

 cd ../../ ; mkdir -p acls/20:00:00:1b:21:37:b8:df/lun_0

 cd acls/20\:00\:00\:1b\:21\:37\:b8\:df/lun_0

 ln -s /sys/kernel/config/target/fc/20\:00\:00\:1b\:21\:55\:1c\:da/tpgt_1/lun/lun_$i/ lun_$i

Here, the WWPN
20:00:00:1b:21:55:1c:da is for the target
20:00:00:1b:21:37:b8:df is for the initiator

 - You can also use block device, such as actual hard disks or ramdisk as the LUNs, the tcm_node command should be adjusted accordingly.  EXample:
  tcm_node --iblock iblock_0/block0 /dev/sdc
  tcm_node --wwn iblock_0/block0
  lio_node --addlun fc-test 1 0 lun0 iblock_0/block0

 - You can create a script to add multiple LUNs to the target, example:
  for i in 1 2 3 4; do
   tcm_node --fileio fileio_0/file$i /home/tgt/file$i 100000000
   tcm_node --wwn fileio_0/file$i
   lio_node --addlun fc-test 1 $i lun0 fileio_0/file$i

 - When running the above commands, a number of warning messages may occur, such as
  "Unable to create iSCSI target logical unit configFS directory"
  "OSError: No such file or directory: /sys/kernel/config/target/iscsi"
  "Unable to disable ISCSI authentication on iSCSI target portal greoup"

  This is because TCM is used by multple targets, include iSCSI and FCoE targets; and some of the codes are not cleaned up properly at this time.

- Fabric configuration
 - There are two methods for fabric configuration: zoning and direct virtual ports connection:
  - Fabric zone:
   - Perform typical zoning; configure the target and the initiator in the same zone. 
   - After zoing, the initiator should be able to access the target (LUNs).
   - You may also need to reset your initiator to have the target/LUNs showing up.

  - Virtual N-port to virtual N-port (back-to-back) connection:
   - run the following commands to create vn-to-vn interface in both initiator and target:
    echo eth2 > /sys/module/libfcoe/parameters/create_vn2vn
   assuming using interface eth2

   - to delete the fcoe interface, run the following commands:
    echo eth2 > /sys/module/libfcoe/parameters/destroy

- Reference:

 - Type "tcm_node -h" to show all tcm_node command parameters

 - Type "lio_node -h" to show all lio_node command parameters
tcm_fcoe_target_guide.txt · Last modified: 2013/01/23 20:35 by rbbratta