The DAQ group distributes a bundle of hardware and software to interested CMS users which allows them to test their interfaces to the DAQ and TCDS system in laboratories before integration in the final system at point 5. The Kit consists of a DTH400 prototype, transceivers and fibres, an MPO-loopback connector for self tests and a 100Gbps TCP/IP network interface card. The software allows to set up the DTH for data acquisition. A simple receiver programme for reception of the DAQ data stream on the receiving PC is provided.
Documentation
The following documents are useful for DTH Kit users:
The DTH Kit Users manual
This manual describes the hardware and provided software of the Kit to get users started.
The DTH_Flashy Users manual
This manual starts the DTH_Flashy utility which allows users to programme the DTH with firmware updates.
Users manual of the SLINKRocket IP cores
This manual is not directly related to the DTH Kit however users will probably want to implement an SLINKRocket in their FEDs to feed data into the FEDKit. This manual describes the usage of the related IP cores.
API documentation of the dthlib
The DTH Controller software is based on a python library called dthib. This library can be used to write custom scripts to configure and monitor the DTH during automised tests. Here you find the API documenation of this library.
GitLab repositories
https://gitlab.cern.ch/cms_daq_dth/fpga_library For reference here follows a list of GitLab repositories which contain sources for the firmware and software used for the DTH Kit. Note that not all code found in these repositories are relevant for the DTH Kit. User should not need to study or checkout these repositories to use the Slink Rocket IPs or the DTH Kit. But for completeness for the interested user the links are listed here.
| Repository | Description |
|---|---|
| CMS_DTH_DAQ | Overall project for the DTH with various sub-projects. The most relevant ones are listed below. |
| DTHKit | Repository with submodules for the DTH-Kit software and documentation |
| CMS_DTH.yum | The yum repository for the DTH software |
| DTH_software | DTH related software packages. Partially used for the DTH-Kit. Make sure the p1_pci branch is selected. |
| SLINKRocket_IPs | The Slink Rocket IPs for Xilinx Ultrascale+ and Virtex+ architectures (sender and receiver core) |
| SLINKRocket_example | An example design using the SlinkRocket IPs, for the Xilinx VCU118 development board. |
| FPGA_libraries | Library of reusable vhdl entities. |
| SLINKRocket | The source code for the Slink Rocket. The base code for the Slink Rocket IPs. |
| dth-p1-linux | Repository with code and documentation on how to create the linux system for the P1 and how to upgrade it (i.e. make a bootable USB stick image) |
Upgrading the P1 Linux system
For the DTH-P1 there is a new P1 Linux system available. It is based on Alma9 Linux and provides an environment similar to the one which is being developed for the P2 (which houses a Zynq based and not comExpress based Controller). This system has been developed to provide users with similar environments in both systems. The new system is a minimal Linux base system including the "podman" utility to run docker containers. Applications are supposed to run in containers. For the DAQUnit there is a first version of such a container available. It provides the DTH DAQUnit software which was also included in the original DTH Kit: DTH_Flashy.py and DTH_Control.py. How to upgrade to this new system is described in the Documenation on gitlab.
Thanks for giving us feedback in case of problems (or also success).
Recovery procedure to the original Linux system
(This is the Linux system which was installed at the time the DTHKits were handed out)
In case your system becomes un-bootable or un-usable we propose the following recovery procedure. (You can also apply this if for some reason the kernel was updated and you do not succeed to boot back into the original kernel version, and you do not succeed to recompile the xdma driver for the IPBUS access. (However: older kernel versions should still be present on your system and should be bootable via grub).
The recovery procedure consists in booting a linux system on a USB stick and then transferring the DTH-KIT "factory image" to the internal ssd memory.
The following is needed to execute the procedure:
- A USB hub
- A USB memory stick with at least 64GB. You will lose all data on the stick.
The following steps should be executed for this:
- If your system still boots recover all files which you might want to use later. You will loose the entire content of the ssd disk on the comExpress board.
- Download the recover image from this CERNBOX link : https://cernbox.cern.ch/s/BpGUBrqAkGZCgTF
- Transfer the image to the USB stick in order to create a bootable USB stick. On linux you can do this with the dd command. You need to be sure which device corresponds to the USB stick you are using (you can inspect the system logs if in doubt).
Make sure that the USB stick IS NOT MOUNTED: many linux systems mount usb sticks automatically.
dd if=dth-recovery.img of=/dev/sd{x} bs=64K conv=fdatasync
x has to be replaced with the letter for your USB stick. (Don't specify a partition but the entire disk, for example, if 'e' is the name of the device which corresponds to the USB stick, specify sde but not sde1 or sde2 ). Be patient here, this might take a long time. - On the back side of the DTH plug the USB hub and into the hub a keyboard, a mouse and the USB stick. Plug a screen into the HDMI input
- NOW POWER CYCLE the DTH (with the handles at the front). This is important otherwise the USB stick will not be recognized during the boot.
- Let the system boot from the USB stick (should be the default).
If this does not work reboot the system (e.g. using the reset button close to the HDMI connector) and press ESC when the first logo is shown on the screen (you only have one second for this). In the BIOS boot sequence have a look if you find the USB device. If you do not then the USB stick is not recognized. You can power cycle again, use a different stick, or a different USB hub. - When the puppy-linux is booted open a terminal (right mouse button -> Utilities -> last menu option for terminal emulator).
- mount the partition of the stick which contains the recovery image:
mkdir recovery
mount /dev/sda3 recovery
cd recovery
You should see a ~30GB large file dth-mmcblk0.img - Transfer the image to the internal ssd memory:
dd if=dth_mmcblk0 of=/dev/mmcblk0 bs=4096 conv=fdatasync
Wait a long time(>30min) for the image to be transferred. - Now you should be able to remove the USB stick and reboot the system in CentOS. You should have the original root and DTH accounts with the original passwords.