Data Server
 Notice : | for requesting a user account, see Common Users Account |
The data server is built for storing all experimental raw data and for analysis. The primary data storage /mnt/data0 (72 TB, RAID 5 array from 6 16 TB disks), should be and only be for raw data storage. Frequent read-write data should be avoided at all costs because it will reduce the disk lifetime. Analysis source code and output should be stored in user's home directory. All necessary analysis codes/programs are provided and maintained by the system admin. Users can also request any programs to be installed or installed in their local.
Network IP and Alias
| IP | network alias | Speed | functions |
|---|---|---|---|
| 128.186.111.42 | pauli.physics.fsu.edu | 1 Gb/s | for user login |
| 128.186.111.43 | pauli-data.physics.fsu.edu | 10 Gb/s | for data transfer |
Spec.
- Dell PowerEdge R540 Rack server
- CPU : Intel Xeon Gold, 36 cores
- RAM : 32 GB
- Storage :
- 1.9TB SSD for OS and /home/ at the rear slot-1
- 1TB for data (temporary) /mnt/data_1TB at the rear slot-2
- 6x16TB for data /mnt/data0 at the front slot-1,2,3,4,5,6
- OS : Ubuntu 20.04
Detail Spec.
| Items | description | Item number |
|---|---|---|
| Additional Network Cards | On-Board LOM | 542-BBBP |
| Additional Processor | No Additional Processor | 374-BBBX |
| Advanced System Configurations | UEFI BIOS Boot Mode with GPT Partition | 800-BBDM |
| Bezel | No Bezel | 350-BBBW |
| Dell EMC Luggage Tag | 350-BBLI | |
| Chassis Configuration | 3.5"Chas up 12 HP HDs 2x3.5" Rear Drives,PERC/HBA11 | 321-BHDM |
| Dell Services: Hardware Support | Basic Hardware Services: Business Hours (5X10) Next Business Day On Site Hardware Warranty Repair 3 Years | 815-4327 |
| Dell Hardware Limited Warranty Plus On-Site Service | 815-4328 | |
| Deployment Services | On-Site Installation Declined | 900-9997 |
| Embedded Systems Management | iDrac9, Express | 385-BBLC |
| Group Manager | iDRAC Group Manager, Disabled | 379-BCQY |
| Hard Drives | 1TB 7.2K RPM SATA 6Gbps 512n 2.5in Hot-plug Hard Drive, 3.5in HYB CARR | 400-ASHG |
| Hard Drives (PCIe SSD/Flex Bay) | 1.92TB SSD SATA Read Intensive 6Gbps 512 2.5in Flex Bay AG Drive,3.5in HYB CARR, 1 DWPD | 400-AXRY |
| Internal Optical Drive | No Internal Optical Drive | 429-AAIQ |
| Memory Capacity | 32GB RDIMM, 3200MT/s, Dual Rank 16Gb BASE x8 | 370-AGDS |
| Memory Configuration Type | Performance Optimized | 370-AAIP |
| Memory DIMM Type and Speed | 3200MT/s RDIMMs | 370-AEVR |
| Motherboard | PowerEdge R540 MLK Motherboard, V2 | 329-BFBK |
| Operating System | No Operating System | 619-ABVR |
| OS Media Kits | No Media Required | 421-5736 |
| Password | iDRAC,Factory Generated Password | 379-BCSF |
| PCIe Riser | 1xFH, 1 or 2CPU | 330-BBHT |
| Power Cords | Power Cord - C13, 3M, 125V, 15A (North America, Guam, North Marianas, Philippines, Samoa, Vietnam) | 450-AALV |
| Power Management BIOS Settings | Performance BIOS Settings | 384-BBBL |
| Power Supply | Single,Hot-plug Power Supply,495W | 450-AGUG |
| Processor | Intel Xeon Gold 5220 2.2G, 18C/36T, 10.4GT/s, 24.75M Cache, Turbo, HT (125W) DDR4-2666 | 338-BSDI |
| Processor Thermal Configuration | 1 CPU Standard Thermal for 3.5" Chassis | 322-BBBJ |
| Quick Sync 2 (At-the-box mgmt) | Quick Sync 2 (At-the-box mgmt) | 350-BBKV |
| Rack Rails | ReadyRails Sliding Rails With Cable Management Arm | 770-BBBR |
| RAID Configuration | Unconfigured RAID | 780-BCDS |
| RAID/Internal Storage Controllers | PERC H750 Adapter, Low Profile | 405-ABCC |
| Regulatory | PowerEdge R540 CCC and BIS Marking, No CE Marking | 389-DSXE |
| Shipping | PowerEdge R540 Shipping | 340-BSID |
| Shipping Information | US Order | 332-1286 |
| Shipping Material | PowerEdge R540 Shipping Material, V2 | 340-COSN |
| System Documentation | No Systems Documentation, No OpenManage DVD Kit | 631-AACK |
| Trusted Platform Module (TPM) | Trusted Platform Module 2.0 | 461-AAEM |
Software
| Software/Code | Info. |
|---|---|
| CERN ROOT 6.26/02 | |
| docker | |
| python3-pip | pip or pip3 are the command, avalible packages |
| PtolemyGUI | GUI for DWBA calculation using Ptolemy, it will create a file call DWBA. |
| /opt/Ptolemy/Cleopatra/Isotope.h | a C++ Class (Isotope) for mass20.txt |
| (TBD) twoBodyReaction.h | a C++ Class for A+a→b+B reaction kinematics. |
| (TBD) Radware | |
| gnuscope (via docker) | Only able to run at local, for user Tandem |
| Globus personal connect |
Storage and Disks
| Disk | Capacity | Speed | Type | Slot | Command |
|---|---|---|---|---|---|
| / | 1.9 TB | 533 MB/s | SSD | rear-1 | OS disk |
| /mnt/data_1TB | 1 TB | 108 MB/s | HDD | rear-2 | not recommended |
| /mnt/data0 | 80 TB | 340 MB/s | RAID-5 6x16TB HHD | front-1,2,3,4,5,6 | recommended |
Common Users Account
tandem (fsuphysics888) is the general user account.
Anyone wish to have a personal account can be contact Ryan mailto:rtang@fsu.edu, and provides following information
- full name
- username (the name for login)
[ admin only ] Create an user account
there is a bash script in /usr/bin/createNewUser.sh. This script will create an account and set disk-quota limit.
User Quota
The default quota for user is 50GB soft limit and 100GB hard limit on their home directory. The quota can be increase on request. User can also use the /mnt/data_1TB. The user quota for raw data saving at /mnt/data0 are unlimited.
- soft limit: user still can use more than 50G in the next 7 days, after that, this 50G is the hard limit.
- hard limit: user can never use more than 100G.
Setup
sudo apt install quota
edit the /etc/fstab
from this
UUID=fc15d3a2-a49f-4974-8216-5949a740ef80 / ext4 errors=remount-ro 0 1
to this:
UUID=fc15d3a2-a49f-4974-8216-5949a740ef80 / ext4 errors=remount-ro,usrquota 0 1
remount
sudo mount -o remount /
check the new effect
cat /proc/mounts | grep ' / '
/dev/sdc2 / ext4 rw,relatime,quota,usrquota</spam>,errors=remount-ro,stripe=64 0 0
enable the quota
sudo quotacheck -ugm /
This will create /aquota.user
check
sudo quotaon -v / /dev/sdc2 [/]: user quotas turned on
Set user quota
use the command 'setquota' to set user quota, in this example 50G is the soft limit, 100G is the hard limit.
sudo setquota -u <User> 50G 100G 0 0 /
quota report
sudo repquota -s /
Globus Personal Connect
| Notice : | Ryan already made an endpoint on the data server called FSU Fox's Lab Data Server Ryan's End Point |
For more detail, please check https://docs.globus.org/how-to/globus-connect-personal-linux/
The program is stored in /opt/globusconnectpersonal/
to Run
~>/opt/globusconnectpersonal/globusconnect &
It will ask you to login, follow the steps and use your FSU credential.
enable access of data0
if you cannot access the data0, edit the file at you home ~/.globusonline/lta/config-path
~/,0,1 /mnt/,1,1
The 1st line is the access of your home directory, not-share, allow read/write.
The 2nd line is the access of the /mnt/ directory, share, allow read/write.
To clear the setting / remove credential
Simply remove the folder ~/.globusonline
Exp Folder format
Please use
YYYY_MM_beam_reaction_PI_extra
- YYYY : year in 4 digits
- MM : month is 4 digits
- beam : e.g. 12C
- reaction : e.g. dp
- PI : name of PI
- extra : some extra comments, can be skipped.
For example, Eli performed 12C(d,n)reaction on April, 2022.
2022_04_12C_dn_eli
Mount disk remotely
To mount the disk remotely, need the sshfs installed in the local machine. In Debian/Ubuntu
~>sudo apt install sshfs
Create a mount directory in the local machine (say /mnt/dataRemote)
~>sudo mkdir /mnt/dataRemote
Mounting
~>sudo sshfs -o allow_other YYY@pauli-data.physics.fsu.edu:XXX /mnt/dataRemote/
where YYY is the user account where XXX is the disk location in the data server, e.g. /mnt/data0
To mount on startup
edit /etc/fstab
add line
ssh#YYY@pauli.physics.fsu.edu:XXX /mnt/dataRemote/
Python3 packages
- apturl 0.5.2
- bcrypt 3.1.7
- blinker 1.4
- Brlapi 0.7.0
- certifi 2019.11.28
- cffi 1.15.0
- chardet 3.0.4
- click 8.1.3
- colorama 0.4.3
- command-not-found 0.3
- cryptography 36.0.2
- cupshelpers 1.0
- dbus-python 1.2.16
- defer 1.0.6
- distro 1.4.0
- distro-info 0.23ubuntu1
- duplicity 0.8.12.0
- entrypoints 0.3
- fasteners 0.14.1
- future 0.18.2
- globus-cli 3.5.0
- globus-sdk 3.8.0
- httplib2 0.14.0
- idna 2.8
- jmespath 0.10.0
- keyring 18.0.1
- language-selector 0.1
- launchpadlib 1.10.13
- lazr.restfulclient 0.14.2
- lazr.uri 1.0.3
- lockfile 0.12.2
- louis 3.12.0
- macaroonbakery 1.3.1
- matplotlib 3.5.2
- monotonic 1.5
- netifaces 0.10.4
- numpy 1.22.3
- oauthlib 3.1.0
- olefile 0.46
- paramiko 2.6.0
- pexpect 4.6.0
- Pillow 7.0.0
- pip 20.0.2
- plotly 5.8.0
- protobuf 3.6.1
- pycairo 1.16.2
- pycparser 2.21
- pycups 1.9.73
- PyGObject 3.36.0
- PyJWT 2.3.0
- pymacaroons 0.13.0
- PyNaCl 1.3.0
- pyRFC3339 1.1
- PySimpleGUI 4.59.0
- python-apt 2.0.0+ubuntu0.20.4.7
- python-dateutil 2.7.3
- python-debian 0.1.36ubuntu1
- pytz 2019.3
- pyxdg 0.26
- PyYAML 5.3.1
- reportlab 3.5.34
- requests 2.22.0
- requests-unixsocket 0.2.0
- SecretStorage 2.3.1
- setuptools 45.2.0
- simplejson 3.16.0
- six 1.14.0
- ssh-import-id 5.10
- systemd-python 234
- typing-extensions 4.2.0
- ubuntu-advantage-tools 27.7
- ubuntu-drivers-common 0.0.0
- ufw 0.36
- unattended-upgrades 0.1
- urllib3 1.25.8
- usb-creator 0.3.7
- wadllib 1.3.3
- wheel 0.34.2
- xkit 0.0.0
UPS
 Warning : | shutdown protection not tested |
A UPS is used for power the data server, the achieve data server, and the Ethernet switches on the rack.
a program for monitoring the UPS is installed from https://www.cyberpowersystems.com/product/software/power-panel-business/powerpanel-business-4-8-1-linux/
The program provide a web interface.
pauli.physics.fsu.edu:3052
for account and password, see elog
Others
Contact
- Ryan Tang mailto:rtang@fsu.edu