Data Server

Notice :

for requesting a user account, see Common Users Account

Photo of the rack of the data server. The data server is on the top. The achive data server is in the middle. The UPS is at the bottom.

The data server is built for storing all experimental raw data and for analysis. The primary data storage /mnt/data0 (72 TB, RAID 5 array from 6 16 TB disks), should be and only be for raw data storage. Frequent read-write data should be avoided at all costs because it will reduce the disk lifetime. Analysis source code and output should be stored in user's home directory. All necessary analysis codes/programs are provided and maintained by the system admin. Users can also request any programs to be installed or installed in their local.

Network IP and Alias

IP	network alias	Speed	functions
128.186.111.42	pauli.physics.fsu.edu	1 Gb/s	for user login
128.186.111.43	pauli-data.physics.fsu.edu	10 Gb/s	for data transfer

Spec.

• Dell PowerEdge R540 Rack server
• CPU : Intel(R) Xeon(R) Gold 5220 CPU @ 2.20GHz (18 core x 2 = 36 threads)
• RAM : 4 X 32 GB (upgraded on May 20, 2024)
• Storage :
  • 1.9TB SSD for OS and /home/ at the rear slot-1
  • 8TB for data (temporary) /mnt/data_8TB at the rear slot-2
  • 6x16TB for data /mnt/data0 at the front slot-1,2,3,4,5,6
• OS : Ubuntu 20.04

Detail Spec.

Here are the detail spec.

Items	description	Item number
Additional Network Cards	On-Board LOM	542-BBBP
Additional Processor	No Additional Processor	374-BBBX
Advanced System Configurations	UEFI BIOS Boot Mode with GPT Partition	800-BBDM
Bezel	No Bezel	350-BBBW
	Dell EMC Luggage Tag	350-BBLI
Chassis Configuration	3.5"Chas up 12 HP HDs 2x3.5" Rear Drives,PERC/HBA11	321-BHDM
Dell Services: Hardware Support	Basic Hardware Services: Business Hours (5X10) Next Business Day On Site Hardware Warranty Repair 3 Years	815-4327
	Dell Hardware Limited Warranty Plus On-Site Service	815-4328
Deployment Services	On-Site Installation Declined	900-9997
Embedded Systems Management	iDrac9, Express	385-BBLC
Group Manager	iDRAC Group Manager, Disabled	379-BCQY
Hard Drives	1TB 7.2K RPM SATA 6Gbps 512n 2.5in Hot-plug Hard Drive, 3.5in HYB CARR	400-ASHG
Hard Drives (PCIe SSD/Flex Bay)	1.92TB SSD SATA Read Intensive 6Gbps 512 2.5in Flex Bay AG Drive,3.5in HYB CARR, 1 DWPD	400-AXRY
Internal Optical Drive	No Internal Optical Drive	429-AAIQ
Memory Capacity	32GB RDIMM, 3200MT/s, Dual Rank 16Gb BASE x8	370-AGDS
Memory Configuration Type	Performance Optimized	370-AAIP
Memory DIMM Type and Speed	3200MT/s RDIMMs DDR4	370-AEVR
Motherboard	PowerEdge R540 MLK Motherboard, V2	329-BFBK
Operating System	No Operating System	619-ABVR
OS Media Kits	No Media Required	421-5736
Password	iDRAC,Factory Generated Password	379-BCSF
PCIe Riser	1xFH, 1 or 2CPU	330-BBHT
Power Cords	Power Cord - C13, 3M, 125V, 15A (North America, Guam, North Marianas, Philippines, Samoa, Vietnam)	450-AALV
Power Management BIOS Settings	Performance BIOS Settings	384-BBBL
Power Supply	Single,Hot-plug Power Supply,495W	450-AGUG
Processor	Intel Xeon Gold 5220 2.2G, 18C/36T, 10.4GT/s, 24.75M Cache, Turbo, HT (125W) DDR4-2666	338-BSDI
Processor Thermal Configuration	1 CPU Standard Thermal for 3.5" Chassis	322-BBBJ
Quick Sync 2 (At-the-box mgmt)	Quick Sync 2 (At-the-box mgmt)	350-BBKV
Rack Rails	ReadyRails Sliding Rails With Cable Management Arm	770-BBBR
RAID Configuration	Unconfigured RAID	780-BCDS
RAID/Internal Storage Controllers	PERC H750 Adapter, Low Profile	405-ABCC
Regulatory	PowerEdge R540 CCC and BIS Marking, No CE Marking	389-DSXE
Shipping	PowerEdge R540 Shipping	340-BSID
Shipping Information	US Order	332-1286
Shipping Material	PowerEdge R540 Shipping Material, V2	340-COSN
System Documentation	No Systems Documentation, No OpenManage DVD Kit	631-AACK
Trusted Platform Module (TPM)	Trusted Platform Module 2.0	461-AAEM

Software

Software/Code	Info.	Run Command
CERN ROOT 6.26/02	if you don't know, probably you are at the wrong place.	root
KSHELL	Large-scale shell model calculation using multi-cores.	kshell_ui.py
PtolemyGUI	GUI for DWBA calculation using Ptolemy, it will create a file call DWBA.	PtolemyGUI
Jupyter notebook
docker
geany	a nice editor, in additional to emacs, vim, and nano	geany
GEANT4 11.0.2	installed path /usr/local, cmake with -DGEANT4_INSTALL_DATA=ON -DGEANT4_USE_OPENGL_XLL=ON
view3Dview	for VMRL vituallization
python3-pip	pip or pip3 are the command, avalible packages
/opt/Ptolemy/Cleopatra/Isotope.h	a C++ Class (Isotope) for mass20.txt
(TBD) twoBodyReaction.h	a C++ Class for A+a→b+B reaction kinematics.
(TBD) Radware
gnuscope (via docker)	Only able to run at local, for user Tandem
Globus personal connect		globusconnect

Storage and Disks

Disk	Capacity	Speed	Type	Slot	Command
/	1.9 TB	533 MB/s	SSD	rear-1	OS disk
/mnt/data_8TB	8 TB	- MB/s	SSD	rear-2	extra analysis storage
/mnt/data0	80 TB	340 MB/s	RAID-5 6x16TB HHD	front-1,2,3,4,5,6	recommended
/mnt/data1	100 TB	340 MB/s	RAID-5 6x20TB HHD	front-7,8,9,10,11,12	installed on 2024-10-11

• RAID information
• RAID Calculator

Common Users Account

tandem is the general user account.

Anyone wish to have a personal account can be contact Ryan mailto:rtang@fsu.edu, and provides following information

• full name
• username (the name for login)

[ admin only ] Create an user account

there is a bash script in /usr/bin/createNewUser.sh. This script will create an account and set disk-quota limit.

User Quota

The default quota for user is 50GB soft limit and 100GB hard limit on their home directory. The quota can be increase on request. User can also use the /mnt/data_1TB. The user quota for raw data saving at /mnt/data0 are unlimited.

• soft limit: user still can use more than 50G in the next 7 days, after that, this 50G is the hard limit.
• hard limit: user can never use more than 100G.

Setup

install the quota program

sudo apt install quota

edit the /etc/fstab

from this

UUID=fc15d3a2-a49f-4974-8216-5949a740ef80 /   ext4    errors=remount-ro 0       1

to this:

UUID=fc15d3a2-a49f-4974-8216-5949a740ef80 /   ext4    errors=remount-ro,usrquota 0       1

remount

sudo mount -o remount /

check the new effect

cat /proc/mounts | grep ' / '
/dev/sdc2 / ext4 rw,relatime,quota,usrquota</spam>,errors=remount-ro,stripe=64 0 0

enable the quota

sudo quotacheck -ugm /

This will create /aquota.user

check

sudo quotaon -v /
/dev/sdc2 [/]: user quotas turned on

Set user quota

use the command 'setquota' to set user quota, in this example 50G is the soft limit, 100G is the hard limit.

sudo setquota -u <User> 50G 100G 0 0 /

quota report

sudo repquota -s /

Globus Endpoint

An endpoint is created for people to access the data.

UUID : 5ba06aee-87ff-11ef-8ebd-4177e580d0c6
Name : FSU Fox's Lab Data Server Ryan's End Point Shared

[the old end point with UUID : 99d0d880-2644-11ee-8193-195c41bc0be4 is deleted]

Globus Personal Connect

Notice :

Ryan already made an endpoint on the data server called FSU Fox's Lab Data Server Ryan's End Point

For more detail, please check https://docs.globus.org/how-to/globus-connect-personal-linux/

The program is stored in /opt/globusconnectpersonal/

to Run

~>/opt/globusconnectpersonal/globusconnect &

It will ask you to login, follow the steps and use your FSU credential.

enable access of data0

if you cannot access the data0, edit the file at you home ~/.globusonline/lta/config-path

~/,0,1
/mnt/,1,1

The 1st line is the access of your home directory, not-share, allow read/write.

The 2nd line is the access of the /mnt/ directory, share, allow read/write.

To clear the setting / remove credential

Simply remove the folder ~/.globusonline

To keep run in background

nohup /opt/globusconnectpersonal/globusconnect -start > output.log 2>&1 &

Exp Folder format

Please use

YYYY_MM_beam_reaction_PI_extra

• YYYY : year in 4 digits
• MM : month is 4 digits
• beam : e.g. 12C
• reaction : e.g. dp
• PI : name of PI
• extra : some extra comments, can be skipped.

For example, Eli performed 12C(d,n)reaction on April, 2022.

2022_04_12C_dn_eli

Mount disk remotely

To mount the disk remotely, need the sshfs installed in the local machine. In Debian/Ubuntu

~>sudo apt install sshfs

Create a mount directory in the local machine (say /mnt/dataRemote)

~>sudo mkdir /mnt/dataRemote

Mounting

~>sudo sshfs -o allow_other YYY@pauli-data.physics.fsu.edu:XXX  /mnt/dataRemote/

where YYY is the user account where XXX is the disk location in the data server, e.g. /mnt/data0

To mount on startup

edit /etc/fstab

add line

ssh#YYY@pauli.physics.fsu.edu:XXX  /mnt/dataRemote/

Python3 packages

Here is the list of the python3 packages on May 6, 2022

• apturl 0.5.2
• bcrypt 3.1.7
• blinker 1.4
• Brlapi 0.7.0
• certifi 2019.11.28
• cffi 1.15.0
• chardet 3.0.4
• click 8.1.3
• colorama 0.4.3
• command-not-found 0.3
• cryptography 36.0.2
• cupshelpers 1.0
• dbus-python 1.2.16
• defer 1.0.6
• distro 1.4.0
• distro-info 0.23ubuntu1
• duplicity 0.8.12.0
• entrypoints 0.3
• fasteners 0.14.1
• future 0.18.2
• globus-cli 3.5.0
• globus-sdk 3.8.0
• httplib2 0.14.0
• idna 2.8
• jmespath 0.10.0
• keyring 18.0.1
• language-selector 0.1
• launchpadlib 1.10.13
• lazr.restfulclient 0.14.2
• lazr.uri 1.0.3
• lockfile 0.12.2
• louis 3.12.0
• macaroonbakery 1.3.1
• matplotlib 3.5.2
• monotonic 1.5
• netifaces 0.10.4
• numpy 1.22.3
• oauthlib 3.1.0
• olefile 0.46
• paramiko 2.6.0
• pexpect 4.6.0
• Pillow 7.0.0
• pip 20.0.2
• plotly 5.8.0
• protobuf 3.6.1
• pycairo 1.16.2
• pycparser 2.21
• pycups 1.9.73
• PyGObject 3.36.0
• PyJWT 2.3.0
• pymacaroons 0.13.0
• PyNaCl 1.3.0
• pyRFC3339 1.1
• PySimpleGUI 4.59.0
• python-apt 2.0.0+ubuntu0.20.4.7
• python-dateutil 2.7.3
• python-debian 0.1.36ubuntu1
• pytz 2019.3
• pyxdg 0.26
• PyYAML 5.3.1
• reportlab 3.5.34
• requests 2.22.0
• requests-unixsocket 0.2.0
• SecretStorage 2.3.1
• setuptools 45.2.0
• simplejson 3.16.0
• six 1.14.0
• ssh-import-id 5.10
• systemd-python 234
• typing-extensions 4.2.0
• ubuntu-advantage-tools 27.7
• ubuntu-drivers-common 0.0.0
• ufw 0.36
• unattended-upgrades 0.1
• urllib3 1.25.8
• usb-creator 0.3.7
• wadllib 1.3.3
• wheel 0.34.2
• xkit 0.0.0

Uninterruptible Power Supply

Warning :

shutdown protection not tested

A uninterruptible Power Supply (UPS) is used for power the data server, the Archive Data Server, and the Ethernet switches on the rack.

a program for monitoring the UPS is installed from https://www.cyberpowersystems.com/product/software/power-panel-business/powerpanel-business-4-8-1-linux/

The program provide a web interface.

pauli.physics.fsu.edu:3052

for account and password, see elog

System monitoring

I use Prometheus + node_exporter. Same in the FSUNUC setup.

Others

• Documents and Manuels
• Overview in webpage
• Left Pannel LED status

Contact

• Ryan Tang mailto:rtang@fsu.edu