Skip to content

Instructions

Flashing Serenity

These are instructions on how to flash firmware onto the Serenity after already sending the compiled firmware package to the Serenity. Commands are customized to settings and paths on local computer to speed up set-up.

1 - SSH into the Serenity

2 - Copy the hls_connections.xml connections file to your firmware package. 

cp /home/cmx/mtd-emp-toolbox/mtd-daq/lpGBTv2_3_SO1_ceacmsfw_250603_1554_ETL/hls_connections.xml <firmware_package_dir>

3 - Setup a new emp toolbox or use existing one

  • NEW: clone the git repo and compile + setup the toolbox 
git clone ssh://git@gitlab.cern.ch:7999/mtd-backend/daq/software/mtd-emp-toolbox.git toolbox

cd toolbox

make -j4

source scripts/env.sh
  • EXISTING: source the environment 
cd toolbox

source scripts/env.sh

cd mtd-daq

Note: It is recommend that your firmware package directory lives under the toolbox/mtd-daq directory you will be using. If it is not then make sure that when you run source scripts/env.sh the path is correct for flashing the firmware

4 - Run flashing script

python3 init_Serenity.py -fpga x0 -dir '/<firmware_package_dir>' -bit  '<bitfile_on_untarred_file.bit>' -xml 'hls_connections.xml'
  • Example:
python3 init_Serenity.py -fpga x0 -dir /home/cmx/mtd-emp-toolbox/mtd-daq/lpGBTv2_3_SO1_ceacmsfw_250603_1554_ETL -bit 'lpGBTv2_3_SO1.bit' -xml 'hls_connections.xml'

Set-up MTD Software

These are instructions on how to start a fresh mtd daq sofware and run avaliable applications.

1 - Clone the mtd daq software repository

git clone --recurse-submodules -b master https://gitlab.cern.ch/mtd-backend/daq/software/mtd-daq.git mtd-daq-etl 
cd mtd-daq-etl

Note: This repository needs to be cloned inside the emp toolbox from above

2 - Source the environment

source ../scripts/env.sh

3 - Set up your path

export PYTHONPATH=<your_mtd_daq_dir>/src:$PYTHONPATH

4 - Run desired application scripts

  • Python: under ./src/mtddaqsw/apps/ run an available application
python -m src/mtddaqsw/apps/<your_app_script>
  • Cpp: under ../toolbox/core/src available application will end in .cxx
    • Compile all .cxx app then run
# Compile
cd ../toolbox/core
make -j4

# Run
./bin/<app_name>.exe

Note: Scripts we have made for testing can be found here: mtd_daq_etl-Tests. You'll need to paste them into the correct directory to run them

Set-up Firmware

These are instructions that set up the firmware and compiles it from scratch. This includes setting up IPBB, cloning the EMP Framework and its dependencies needed for the MTD DAQ.

Note: If you have updated an existing firmware set-up skip to step 8 for compiling, packaging and sending

1 - Set up Vivado

vivado212

or 

source /home/software/Xilinx/Vivado/2021.2/settings64.sh

2 - Activate IPBB env

cd HackathonVI/

source ipbb-dev-2024b/env.sh

3 - Create working directory

cd /home/mtd/Documents/etl
ipbb init etl-fwk
cd etl-fwk

4 - Clone all needed repositories 

ipbb add git https://:@gitlab.cern.ch:8443/mtd-backend/daq/firmware/emp-fwk.git
ipbb add git https://gitlab.cern.ch/ttc/legacy_ttc.git -b v2.1
ipbb add git https://:@gitlab.cern.ch:8443/cms-tcds/cms-tcds2-firmware.git -b v0_1_1
ipbb add git https://gitlab.cern.ch/HPTD/tclink.git -r fda0bcf07c501f81daeec1421ffdfb46f828f823
ipbb add git https://gitlab.cern.ch/dth_p1-v2/slinkrocket_ips.git -b v03.12
ipbb add git https://:@gitlab.cern.ch:8443/dth_p1-v2/slinkrocket.git -b v03.12
ipbb add git https://github.com/ipbus/ipbus-firmware -b v1.14.1
ipbb add git https://gitlab.cern.ch/gbt-fpga/gbt-fpga.git -b gbt_fpga_6_1_0
ipbb add git https://gitlab.cern.ch/gbt-fpga/lpgbt-fpga.git -b v.2.1
ipbb add git https://:@gitlab.cern.ch:8443/gbtsc-fpga-support/gbt-sc.git -b gbt_sc_4_3

In the end you should have 10 directories in the src directory - Run this command to check ls src/ | wc -l = 10

Note: if this step fails run the following and login: kinit your_user@CERN.CH

5 - Build the project based on the correct Top file

ipbb proj create vivado etl_rbv3 emp-fwk:projects/examples/serenity/dc_ku15p so1_v1/top_lpgbt.dep

6 - Export the correct path 

export PATH=/opt/cactus/bin/uhal/tools:$PATH LD_LIBRARY_PATH=/opt/cactus/lib:$LD_LIBRARY_PATH

7 - Change configuration of the lpGBTs being built

To speed up build or debugging purposes go into src/emp-fwk/projects/examples/serenity/dc_ku15p/firmware/hdl/so1/emp_project_decl_lpgbt.vhd and comment out as many lines as needed that look like the following:

<num>  => (lpgbt, buf, no_fmt, buf, lpgbt),

...

<num> => ( 0=>(false, false, 1, false, lpgbtv1), 1=>(false, false, 1, false, lpgbtv1), 2=>(false, false, 1, false, lpgbtv1), 3=>(false, false, 1, false, lpgbtv1)),


...

<num>  => (FEC12, DATARATE_10G24, PCS),

Note: Also make sure that all lpGBT configs for this file are building with FEC12 and not FEC5 if testing with a RBv3

8 - Compile and package the firmware

Note: If you need to make changes to the firmware, insert new firmware blocks or change default settings/generics do that now before running these commands.

cd proj/etl_rbv3/

ipbb ipbus gendecoders
ipbb vivado generate-project --single
ipbb vivado synth -j4 impl -j4
ipbb vivado package

9 - Send the firmware package to Serenity

DB Instance

Creation

These are instructions on how to take an existing Postgres DB from a server and create a copy of it on another server.

Prerequisite: postgresql version 16.9 is needed, download link

1 - Dump Exisiting Database in old server

pg_dump -U postgres -h localhost -F c -b -v -f mtddb.dump mtdconfigurationdb

This command will create a mtddb.dump file

2 - Send .dump file to new server

# On the new server run:
rsync -azhvP --inplace <user>@old_server:<dir_path>/mtddb.dump .

This command sends the file to the current directory you are in

3 - Create new DB in new server

createdb -U postgres -h localhost -p 5432 mtdconfigurationdb

4 - Open the dump file in the DB

pg_restore -U postgres -h localhost -p 5432 --no-acl --no-owner -d mtdconfigurationdb -v mtddb.dump

5 - Login to DB

psql -U postgres -d mtdconfigurationdb -h 127.0.0.1 -W

From then onward you can query and use the DB as you wish. For more information on how to use a Postgres SQL Database check out PostgesSQL Documentation

Deletion

How to stop and delete a Postgres SQL Database running on a server

dropdb -U postgres -h 127.0.0.1 -W mtdconfigurationdb

Login

How to login to test instance database once connected to a server. 

psql -U postgres -d mtdconfigurationdb -h 127.0.0.1 -W