Data Server: Difference between revisions

From FSU Fox's Lab Wiki
Jump to navigation Jump to search
 
(115 intermediate revisions by the same user not shown)
Line 1: Line 1:
{{Notice | for requesting a user account, see [[Data_Server#Common_Users_Account | Common Users Account]]}}
[[File:Photo of the rack of the data server.png|thumb|right|Photo of the rack of the data server. The data server is on the top. The achive data server is in the middle. The UPS is at the bottom.]]
The data server is built for storing all experimental raw data and for analysis. The primary data storage /mnt/data0 (72 TB, RAID 5 array from 6 16 TB disks), should be and only be for raw data storage. Frequent read-write data should be avoided at all costs because it will reduce the disk lifetime. Analysis source code and output should be stored in user's home directory. All necessary analysis codes/programs are provided and maintained by the system admin. Users can also request any programs to be installed or installed in their local.
= Network IP and Alias =
= Network IP and Alias =


128.186.111.42
{| class='wikitable'
 
! IP || network alias || Speed || functions
pauli.physics.fsu.edu
|-
| 128.186.111.42 || pauli.physics.fsu.edu || 1 Gb/s || for user login
|-
| 128.186.111.43 || pauli-data.physics.fsu.edu || 10 Gb/s || for data transfer
|}


= Spec. =  
= Spec. =  
 
* Dell PowerEdge R540 Rack server
* CPU : Intel Xeon Gold, 36 cores
* CPU : Intel(R) Xeon(R) Gold 5220 CPU @ 2.20GHz (18 core x 2 = 36 threads)
* RAM : 32 GB
* RAM : 4 X 32 GB (upgraded on May 20, 2024)
* Storage :  
* Storage :  
** 1.9TB SSD for OS and /home/
** 1.9TB SSD for OS and /home/ at the rear slot-1
** 1TB for data (temporary) /mnt/data_1TB
** 8TB for data (temporary) /mnt/data_8TB at the rear slot-2
** 3x16TB for data /mnt/data0
** 6x16TB for data /mnt/data0 at the front slot-1,2,3,4,5,6
* OS : Ubuntu 20.04
* OS : Ubuntu 20.04


= Software =
[[File:Power of the DataServer.png|600px|frameless|none]]


* CERN ROOT 6.26/02
=== Detail Spec. ===
* docker
<div class="toccolours mw-collapsible mw-collapsed">
* (TBD)Ptolemy (DWBA calculation)  
<div style="line-height:1.6;"> Here are the detail spec. </div>
* (TBD)Radware
<div class="mw-collapsible-content">
* (TBD)gonscope (via docker)
{|class='wikitable'
!Items || description || Item number
|-
|Additional Network Cards || On-Board LOM || 542-BBBP
|-
|Additional Processor || No Additional Processor || 374-BBBX
|-
|Advanced System Configurations || UEFI BIOS Boot Mode with GPT Partition || 800-BBDM
|-
|Bezel || No Bezel || 350-BBBW
|-
| || Dell EMC Luggage Tag || 350-BBLI
|-
|Chassis Configuration || 3.5"Chas up 12 HP HDs 2x3.5" Rear Drives,PERC/HBA11 || 321-BHDM
|-
|Dell Services: Hardware Support  || Basic Hardware Services: Business Hours (5X10) Next Business Day On Site Hardware Warranty Repair 3 Years || 815-4327
|-
| || Dell Hardware Limited Warranty Plus On-Site Service || 815-4328
|-
|Deployment Services || On-Site Installation Declined || 900-9997
|-
|Embedded Systems Management ||iDrac9, Express || 385-BBLC
|-
|Group Manager || iDRAC Group Manager, Disabled || 379-BCQY
|-
|Hard Drives || 1TB 7.2K RPM SATA 6Gbps 512n 2.5in Hot-plug Hard Drive, 3.5in HYB CARR || 400-ASHG
|-
|Hard Drives (PCIe SSD/Flex Bay) || 1.92TB SSD SATA Read Intensive 6Gbps 512 2.5in Flex Bay AG Drive,3.5in HYB CARR, 1 DWPD || 400-AXRY
|-
|Internal Optical Drive || No Internal Optical Drive || 429-AAIQ
|-
|Memory Capacity || 32GB RDIMM, 3200MT/s, Dual Rank 16Gb BASE x8 || 370-AGDS
|-
|Memory Configuration Type || Performance Optimized || 370-AAIP
|-
|Memory DIMM Type and Speed || 3200MT/s RDIMMs DDR4 || 370-AEVR
|-
|Motherboard || PowerEdge R540 MLK Motherboard, V2 || 329-BFBK
|-
|Operating System || No Operating System || 619-ABVR
|-
|OS Media Kits || No Media Required || 421-5736
|-
|Password || iDRAC,Factory Generated Password || 379-BCSF
|-
|PCIe Riser || 1xFH, 1 or 2CPU || 330-BBHT
|-
|Power Cords || Power Cord - C13, 3M, 125V, 15A (North America, Guam, North Marianas, Philippines, Samoa, Vietnam) || 450-AALV
|-
|Power Management BIOS Settings || Performance BIOS Settings || 384-BBBL
|-
|Power Supply || Single,Hot-plug Power Supply,495W || 450-AGUG
|-
|Processor || Intel Xeon Gold 5220 2.2G, 18C/36T, 10.4GT/s, 24.75M Cache, Turbo, HT (125W) DDR4-2666 || 338-BSDI
|-
|Processor Thermal Configuration || 1 CPU Standard Thermal for 3.5" Chassis || 322-BBBJ
|-
|Quick Sync 2 (At-the-box mgmt) || Quick Sync 2 (At-the-box mgmt) ||350-BBKV
|-
|Rack Rails || ReadyRails Sliding Rails With Cable Management Arm || 770-BBBR
|-
|RAID Configuration || Unconfigured RAID || 780-BCDS
|-
|RAID/Internal Storage Controllers || PERC H750 Adapter, Low Profile || 405-ABCC
|-
|Regulatory || PowerEdge R540 CCC and BIS Marking, No CE Marking || 389-DSXE
|-
|Shipping || PowerEdge R540 Shipping || 340-BSID
|-
|Shipping Information || US Order || 332-1286
|-
|Shipping Material || PowerEdge R540 Shipping Material, V2 || 340-COSN
|-
|System Documentation || No Systems Documentation, No OpenManage DVD Kit || 631-AACK
|-
|Trusted Platform Module (TPM) || Trusted Platform Module 2.0 || 461-AAEM
|}
</div></div>


= Common Users Account =
= Software =  


'''tandem''' (fsuphysics888) is the general user account.  
{|class='wikitable'
 
! Software/Code !! Info. !! Run Command
Anyone wish to have a personal account can be contact Ryan mailto:rtang@fsu.edu, and provides following information
|-
* full name:
| [https://root.cern/ CERN ROOT 6.26/02] || if you don't know, probably you are at the wrong place. || root
* username:
|-
* purpose:
| [https://sites.google.com/alumni.tsukuba.ac.jp/kshell-nuclear/ KSHELL] || Large-scale shell model calculation using multi-cores. || kshell_ui.py
|-
| PtolemyGUI || GUI for DWBA calculation using Ptolemy, it will create a file call DWBA. || PtolemyGUI
|-
| [https://jupyter.org/ Jupyter notebook] ||
|-
| [https://www.docker.com/ docker] ||
|-
| [https://www.geany.org/ geany] || a nice editor, in additional to emacs, vim, and nano || geany
|-
| [https://geant4.web.cern.ch/support/user_documentation GEANT4 11.0.2] || installed path /usr/local, cmake with -DGEANT4_INSTALL_DATA=ON -DGEANT4_USE_OPENGL_XLL=ON
|-
| view3Dview || for VMRL vituallization
|-
| python3-pip || '''pip''' or '''pip3''' are the command, [[Data_Server#Python3_packages | avalible packages]]
|-
| /opt/Ptolemy/Cleopatra/Isotope.h || a C++ Class (Isotope) for mass20.txt
|-
| <span style="color:grey">(TBD) twoBodyReaction.h</span> || a C++ Class for A+a→b+B reaction kinematics.
|-
| <span style="color:grey">(TBD) Radware ||
|-
| [[gnuscope]] (via docker) || Only able to run at local, for user Tandem
|-
| Globus personal connect || || globusconnect
|}


= Disks available =  
= Storage and Disks =  


{|class='wikitable' style='width 400px'
{|class='wikitable' style='width 400px'
! Disk  !! Capacity !! Type !! Command
! Disk  !! Capacity !! Speed !!  Type !! Slot  !!Command
|-
| / || 1.9 TB || 533 MB/s || SSD || rear-1 || OS disk
|-
| /mnt/data_8TB || 8 TB || - MB/s|| SSD || rear-2  || extra analysis storage
|-
|-
| /mnt/data_1TB || 1 TB || HDD || not recommended
| /mnt/data0    || 80 TB || 340 MB/s || RAID-5 6x16TB HHD || front-1,2,3,4,5,6  || recommended
|-
|-
| /mnt/data0   || 48 TB || RAID-0 3x16TB HHD || recommended
| /mnt/data1   || 100 TB || 340 MB/s || RAID-5 6x20TB HHD || front-7,8,9,10,11,12  || installed on 2024-10-11
|}
|}


*[https://www.dell.com/support/kbdoc/en-ba/000128635/dell-servers-what-are-the-raid-levels-and-their-specifications?lang=en#c RAID information ]
*[https://www.dell.com/support/kbdoc/en-ba/000128635/dell-servers-what-are-the-raid-levels-and-their-specifications?lang=en#c RAID information ]
*[https://www.seagate.com/internal-hard-drives/raid-calculator/ RAID Calculator]
*[https://www.seagate.com/internal-hard-drives/raid-calculator/ RAID Calculator]
= Common Users Account =
'''tandem''' is the general user account.
Anyone wish to have a personal account can be contact Ryan mailto:rtang@fsu.edu, and provides following information
* full name
* username (the name for login)
== <nowiki>[</nowiki> admin only <nowiki>]</nowiki> Create an user account ==
there is a bash script in /usr/bin/createNewUser.sh. This script will create an account and set disk-quota limit.
= User Quota =
The default quota for user is <span style="color:BlueViolet">50GB</span> soft limit and <span style="color:BlueViolet">100GB</span> hard limit on their home directory. The quota can be increase on request. User can also use the /mnt/data_1TB. The user quota for raw data saving at /mnt/data0 are unlimited.
*'''soft limit''': user still can use more than 50G in the next 7 days, after that, this 50G is the hard limit.
*'''hard limit''': user can never use more than 100G.
=== Setup ===
<div class="toccolours mw-collapsible mw-collapsed">
<div style="line-height:1.6;">install the <b>quota</b> program</div>
<div class="mw-collapsible-content">
sudo apt install quota
==== edit the /etc/fstab ====
from this
UUID=fc15d3a2-a49f-4974-8216-5949a740ef80 /  ext4    errors=remount-ro 0      1
to this:
UUID=fc15d3a2-a49f-4974-8216-5949a740ef80 /  ext4    errors=remount-ro,<span style="color:blue">usrquota</span> 0      1
remount
sudo mount -o remount /
check the new effect
cat /proc/mounts | grep ' / '
/dev/sdc2 / ext4 rw,relatime,quota,<span style="color:blue">usrquota</spam>,errors=remount-ro,stripe=64 0 0
==== enable the quota ====
sudo quotacheck -ugm /
This will create '''/aquota.user'''
check
sudo quotaon -v /
/dev/sdc2 [/]: user quotas turned on
==== Set user quota ====
use the command 'setquota' to set user quota, in this example 50G is the soft limit, 100G is the hard limit.
sudo setquota -u <User> 50G 100G 0 0 /
quota report
sudo repquota -s /
</div></div>
= Globus Endpoint =
An endpoint is created for people to access the data.
UUID : 5ba06aee-87ff-11ef-8ebd-4177e580d0c6
Name : FSU Fox's Lab Data Server Ryan's End Point Shared
[the old end point with UUID : 99d0d880-2644-11ee-8193-195c41bc0be4 is deleted]
= Globus Personal Connect =
{{Notice| Ryan already made an endpoint on the data server called '''FSU Fox's Lab Data Server Ryan's End Point'''}}
For more detail, please check https://docs.globus.org/how-to/globus-connect-personal-linux/
The program is stored in '''/opt/globusconnectpersonal/'''
==== to Run ====
~>/opt/globusconnectpersonal/globusconnect &
It will ask you to login, follow the steps and use your FSU credential.
===== enable access of data0 =====
if you cannot access the data0, edit the file at you home '''~/.globusonline/lta/config-path'''
~/,0,1
/mnt/,1,1
The 1st line is the access of your home directory, not-share, allow read/write.
The 2nd line is the access of the /mnt/ directory, share, allow read/write.
==== To clear the setting / remove credential ====
Simply remove the folder '''~/.globusonline'''
==== To keep run in background ====
nohup /opt/globusconnectpersonal/globusconnect -start > output.log 2>&1 &
= Exp Folder format =
Please use
YYYY_MM_beam_reaction_PI_extra
* YYYY : year in 4 digits
* MM : month is 4 digits
* beam : e.g. 12C
* reaction : e.g. dp
* PI : name of PI
* extra : some extra comments, can be skipped.
For example, Eli performed 12C(d,n)reaction on April, 2022.
2022_04_12C_dn_eli


= Mount disk remotely =  
= Mount disk remotely =  
Line 50: Line 282:
  ~>sudo apt install sshfs
  ~>sudo apt install sshfs


Create a mount directory in the local machine (say /mnt/data1)
Create a mount directory in the local machine (say /mnt/dataRemote)
  ~>sudo mkdir /mnt/data1
  ~>sudo mkdir /mnt/dataRemote


Mounting
Mounting
  ~>sudo sshfs -o allow_other <span style="color:blue">YYY</span>@pauli.physics.fsu.edu:<span style="color:blue">XXX</span>  /mnt/data1/
  ~>sudo sshfs -o allow_other <span style="color:blue">YYY</span>@<span style="color:red">pauli-data</span>.physics.fsu.edu:<span style="color:blue">XXX</span>  /mnt/dataRemote/


where YYY is the user account
where YYY is the user account
where XXX is the [[ Data Server#Disks available |disk location]] in the data server.
where XXX is the [[ Data Server#Disks available |disk location]] in the data server, e.g. /mnt/data0


==== To mount on startup ====
==== To mount on startup ====
Line 65: Line 297:
add line
add line


  ssh#YYY@pauli.physics.fsu.edu:XXX  /mnt/data1/
  ssh#YYY@pauli.physics.fsu.edu:XXX  /mnt/dataRemote/
 
= Python3 packages =
 
<div class="toccolours mw-collapsible mw-collapsed">
<div style="line-height:1.6;">Here is the list of the python3 packages on May 6, 2022 </div>
<div class="mw-collapsible-content">
*apturl                0.5.2             
*bcrypt                3.1.7             
*blinker                1.4               
*Brlapi                0.7.0             
*certifi                2019.11.28         
*cffi                  1.15.0             
*chardet                3.0.4             
*click                  8.1.3             
*colorama              0.4.3             
*command-not-found      0.3               
*cryptography          36.0.2             
*cupshelpers            1.0               
*dbus-python            1.2.16             
*defer                  1.0.6             
*distro                1.4.0             
*distro-info            0.23ubuntu1       
*duplicity              0.8.12.0           
*entrypoints            0.3               
*fasteners              0.14.1             
*future                0.18.2             
*globus-cli            3.5.0             
*globus-sdk            3.8.0             
*httplib2              0.14.0             
*idna                  2.8               
*jmespath              0.10.0             
*keyring                18.0.1             
*language-selector      0.1               
*launchpadlib          1.10.13           
*lazr.restfulclient    0.14.2             
*lazr.uri              1.0.3             
*lockfile              0.12.2             
*louis                  3.12.0             
*macaroonbakery        1.3.1   
*matplotlib            3.5.2           
*monotonic              1.5               
*netifaces              0.10.4             
*numpy                  1.22.3             
*oauthlib              3.1.0             
*olefile                0.46               
*paramiko              2.6.0             
*pexpect                4.6.0             
*Pillow                7.0.0             
*pip                    20.0.2
*plotly                5.8.0           
*protobuf              3.6.1             
*pycairo                1.16.2             
*pycparser              2.21               
*pycups                1.9.73             
*PyGObject              3.36.0             
*PyJWT                  2.3.0             
*pymacaroons            0.13.0             
*PyNaCl                1.3.0             
*pyRFC3339              1.1               
*PySimpleGUI            4.59.0             
*python-apt            2.0.0+ubuntu0.20.4.7
*python-dateutil        2.7.3             
*python-debian          0.1.36ubuntu1     
*pytz                  2019.3             
*pyxdg                  0.26               
*PyYAML                5.3.1             
*reportlab              3.5.34             
*requests              2.22.0             
*requests-unixsocket    0.2.0             
*SecretStorage          2.3.1             
*setuptools            45.2.0             
*simplejson            3.16.0             
*six                    1.14.0             
*ssh-import-id          5.10               
*systemd-python        234               
*typing-extensions      4.2.0             
*ubuntu-advantage-tools 27.7               
*ubuntu-drivers-common  0.0.0             
*ufw                    0.36               
*unattended-upgrades    0.1               
*urllib3                1.25.8             
*usb-creator            0.3.7             
*wadllib                1.3.3             
*wheel                  0.34.2             
*xkit                  0.0.0   
</div></div>
 
= Uninterruptible Power Supply =
 
{{Warning | shutdown protection not tested }}
 
A uninterruptible Power Supply (UPS) is used for power the data server, the [[Archive Data Server]], and the Ethernet switches on the rack.
 
a program for monitoring the UPS is installed from https://www.cyberpowersystems.com/product/software/power-panel-business/powerpanel-business-4-8-1-linux/
 
The program provide a web interface.
 
pauli.physics.fsu.edu:3052
 
for account and password, see [https://fsunuc.physics.fsu.edu/elog/dataServer/6 elog]
 
= System monitoring =
 
I use Prometheus + node_exporter. Same in the [[Internal:Web_Server#Prometheus | FSUNUC ]] setup.
 
= Others =
 
*[https://www.dell.com/support/home/en-us/product-support/product/poweredge-r540/overview Documents and Manuels]
*[https://www.dell.com/support/manuals/en-us/poweredge-r540/per540_ism_pub/dell-emc-poweredge-r540-overview?guid=guid-a75f5ce9-6d03-4b70-861b-1028858ef1fe&lang=en-us Overview in webpage]
*[https://www.dell.com/support/manuals/en-us/poweredge-r540/per540_ism_pub/status-led-indicators?guid=guid-c18ad1b2-ec7a-4427-b677-4a5bd5f7789d&lang=en-us Left Pannel LED status]
 
= Contact =
* Ryan Tang mailto:rtang@fsu.edu

Latest revision as of 13:37, 11 October 2024

Notice Notice :   for requesting a user account, see Common Users Account
Photo of the rack of the data server. The data server is on the top. The achive data server is in the middle. The UPS is at the bottom.

The data server is built for storing all experimental raw data and for analysis. The primary data storage /mnt/data0 (72 TB, RAID 5 array from 6 16 TB disks), should be and only be for raw data storage. Frequent read-write data should be avoided at all costs because it will reduce the disk lifetime. Analysis source code and output should be stored in user's home directory. All necessary analysis codes/programs are provided and maintained by the system admin. Users can also request any programs to be installed or installed in their local.

Network IP and Alias

IP network alias Speed functions
128.186.111.42 pauli.physics.fsu.edu 1 Gb/s for user login
128.186.111.43 pauli-data.physics.fsu.edu 10 Gb/s for data transfer

Spec.

  • Dell PowerEdge R540 Rack server
  • CPU : Intel(R) Xeon(R) Gold 5220 CPU @ 2.20GHz (18 core x 2 = 36 threads)
  • RAM : 4 X 32 GB (upgraded on May 20, 2024)
  • Storage :
    • 1.9TB SSD for OS and /home/ at the rear slot-1
    • 8TB for data (temporary) /mnt/data_8TB at the rear slot-2
    • 6x16TB for data /mnt/data0 at the front slot-1,2,3,4,5,6
  • OS : Ubuntu 20.04
Power of the DataServer.png

Detail Spec.

Here are the detail spec.
Items description Item number
Additional Network Cards On-Board LOM 542-BBBP
Additional Processor No Additional Processor 374-BBBX
Advanced System Configurations UEFI BIOS Boot Mode with GPT Partition 800-BBDM
Bezel No Bezel 350-BBBW
Dell EMC Luggage Tag 350-BBLI
Chassis Configuration 3.5"Chas up 12 HP HDs 2x3.5" Rear Drives,PERC/HBA11 321-BHDM
Dell Services: Hardware Support Basic Hardware Services: Business Hours (5X10) Next Business Day On Site Hardware Warranty Repair 3 Years 815-4327
Dell Hardware Limited Warranty Plus On-Site Service 815-4328
Deployment Services On-Site Installation Declined 900-9997
Embedded Systems Management iDrac9, Express 385-BBLC
Group Manager iDRAC Group Manager, Disabled 379-BCQY
Hard Drives 1TB 7.2K RPM SATA 6Gbps 512n 2.5in Hot-plug Hard Drive, 3.5in HYB CARR 400-ASHG
Hard Drives (PCIe SSD/Flex Bay) 1.92TB SSD SATA Read Intensive 6Gbps 512 2.5in Flex Bay AG Drive,3.5in HYB CARR, 1 DWPD 400-AXRY
Internal Optical Drive No Internal Optical Drive 429-AAIQ
Memory Capacity 32GB RDIMM, 3200MT/s, Dual Rank 16Gb BASE x8 370-AGDS
Memory Configuration Type Performance Optimized 370-AAIP
Memory DIMM Type and Speed 3200MT/s RDIMMs DDR4 370-AEVR
Motherboard PowerEdge R540 MLK Motherboard, V2 329-BFBK
Operating System No Operating System 619-ABVR
OS Media Kits No Media Required 421-5736
Password iDRAC,Factory Generated Password 379-BCSF
PCIe Riser 1xFH, 1 or 2CPU 330-BBHT
Power Cords Power Cord - C13, 3M, 125V, 15A (North America, Guam, North Marianas, Philippines, Samoa, Vietnam) 450-AALV
Power Management BIOS Settings Performance BIOS Settings 384-BBBL
Power Supply Single,Hot-plug Power Supply,495W 450-AGUG
Processor Intel Xeon Gold 5220 2.2G, 18C/36T, 10.4GT/s, 24.75M Cache, Turbo, HT (125W) DDR4-2666 338-BSDI
Processor Thermal Configuration 1 CPU Standard Thermal for 3.5" Chassis 322-BBBJ
Quick Sync 2 (At-the-box mgmt) Quick Sync 2 (At-the-box mgmt) 350-BBKV
Rack Rails ReadyRails Sliding Rails With Cable Management Arm 770-BBBR
RAID Configuration Unconfigured RAID 780-BCDS
RAID/Internal Storage Controllers PERC H750 Adapter, Low Profile 405-ABCC
Regulatory PowerEdge R540 CCC and BIS Marking, No CE Marking 389-DSXE
Shipping PowerEdge R540 Shipping 340-BSID
Shipping Information US Order 332-1286
Shipping Material PowerEdge R540 Shipping Material, V2 340-COSN
System Documentation No Systems Documentation, No OpenManage DVD Kit 631-AACK
Trusted Platform Module (TPM) Trusted Platform Module 2.0 461-AAEM

Software

Software/Code Info. Run Command
CERN ROOT 6.26/02 if you don't know, probably you are at the wrong place. root
KSHELL Large-scale shell model calculation using multi-cores. kshell_ui.py
PtolemyGUI GUI for DWBA calculation using Ptolemy, it will create a file call DWBA. PtolemyGUI
Jupyter notebook
docker
geany a nice editor, in additional to emacs, vim, and nano geany
GEANT4 11.0.2 installed path /usr/local, cmake with -DGEANT4_INSTALL_DATA=ON -DGEANT4_USE_OPENGL_XLL=ON
view3Dview for VMRL vituallization
python3-pip pip or pip3 are the command, avalible packages
/opt/Ptolemy/Cleopatra/Isotope.h a C++ Class (Isotope) for mass20.txt
(TBD) twoBodyReaction.h a C++ Class for A+a→b+B reaction kinematics.
(TBD) Radware
gnuscope (via docker) Only able to run at local, for user Tandem
Globus personal connect globusconnect

Storage and Disks

Disk Capacity Speed Type Slot Command
/ 1.9 TB 533 MB/s SSD rear-1 OS disk
/mnt/data_8TB 8 TB - MB/s SSD rear-2 extra analysis storage
/mnt/data0 80 TB 340 MB/s RAID-5 6x16TB HHD front-1,2,3,4,5,6 recommended
/mnt/data1 100 TB 340 MB/s RAID-5 6x20TB HHD front-7,8,9,10,11,12 installed on 2024-10-11

Common Users Account

tandem is the general user account.

Anyone wish to have a personal account can be contact Ryan mailto:rtang@fsu.edu, and provides following information

  • full name
  • username (the name for login)

[ admin only ] Create an user account

there is a bash script in /usr/bin/createNewUser.sh. This script will create an account and set disk-quota limit.

User Quota

The default quota for user is 50GB soft limit and 100GB hard limit on their home directory. The quota can be increase on request. User can also use the /mnt/data_1TB. The user quota for raw data saving at /mnt/data0 are unlimited.

  • soft limit: user still can use more than 50G in the next 7 days, after that, this 50G is the hard limit.
  • hard limit: user can never use more than 100G.

Setup

install the quota program
sudo apt install quota

edit the /etc/fstab

from this 

UUID=fc15d3a2-a49f-4974-8216-5949a740ef80 /   ext4    errors=remount-ro 0       1

to this: 

UUID=fc15d3a2-a49f-4974-8216-5949a740ef80 /   ext4    errors=remount-ro,usrquota 0       1

remount 

sudo mount -o remount /

check the new effect 

cat /proc/mounts | grep ' / '
/dev/sdc2 / ext4 rw,relatime,quota,usrquota</spam>,errors=remount-ro,stripe=64 0 0

enable the quota

sudo quotacheck -ugm /

This will create /aquota.user

check 

sudo quotaon -v /
/dev/sdc2 [/]: user quotas turned on

Set user quota

use the command 'setquota' to set user quota, in this example 50G is the soft limit, 100G is the hard limit. 

sudo setquota -u <User> 50G 100G 0 0 /

quota report 

sudo repquota -s /

Globus Endpoint

An endpoint is created for people to access the data. 

UUID : 5ba06aee-87ff-11ef-8ebd-4177e580d0c6
Name : FSU Fox's Lab Data Server Ryan's End Point Shared

[the old end point with UUID : 99d0d880-2644-11ee-8193-195c41bc0be4 is deleted]

Globus Personal Connect

Notice Notice :   Ryan already made an endpoint on the data server called FSU Fox's Lab Data Server Ryan's End Point

For more detail, please check https://docs.globus.org/how-to/globus-connect-personal-linux/

The program is stored in /opt/globusconnectpersonal/

to Run

~>/opt/globusconnectpersonal/globusconnect &

It will ask you to login, follow the steps and use your FSU credential.

enable access of data0

if you cannot access the data0, edit the file at you home ~/.globusonline/lta/config-path 

~/,0,1
/mnt/,1,1

The 1st line is the access of your home directory, not-share, allow read/write.

The 2nd line is the access of the /mnt/ directory, share, allow read/write.

To clear the setting / remove credential

Simply remove the folder ~/.globusonline

To keep run in background

nohup /opt/globusconnectpersonal/globusconnect -start > output.log 2>&1 &

Exp Folder format

Please use 

YYYY_MM_beam_reaction_PI_extra
  • YYYY : year in 4 digits
  • MM : month is 4 digits
  • beam : e.g. 12C
  • reaction : e.g. dp
  • PI : name of PI
  • extra : some extra comments, can be skipped.

For example, Eli performed 12C(d,n)reaction on April, 2022. 

2022_04_12C_dn_eli

Mount disk remotely

To mount the disk remotely, need the sshfs installed in the local machine. In Debian/Ubuntu 

~>sudo apt install sshfs

Create a mount directory in the local machine (say /mnt/dataRemote) 

~>sudo mkdir /mnt/dataRemote

Mounting 

~>sudo sshfs -o allow_other YYY@pauli-data.physics.fsu.edu:XXX  /mnt/dataRemote/

where YYY is the user account where XXX is the disk location in the data server, e.g. /mnt/data0

To mount on startup

edit /etc/fstab

add line 

ssh#YYY@pauli.physics.fsu.edu:XXX  /mnt/dataRemote/

Python3 packages

Here is the list of the python3 packages on May 6, 2022
  • apturl 0.5.2
  • bcrypt 3.1.7
  • blinker 1.4
  • Brlapi 0.7.0
  • certifi 2019.11.28
  • cffi 1.15.0
  • chardet 3.0.4
  • click 8.1.3
  • colorama 0.4.3
  • command-not-found 0.3
  • cryptography 36.0.2
  • cupshelpers 1.0
  • dbus-python 1.2.16
  • defer 1.0.6
  • distro 1.4.0
  • distro-info 0.23ubuntu1
  • duplicity 0.8.12.0
  • entrypoints 0.3
  • fasteners 0.14.1
  • future 0.18.2
  • globus-cli 3.5.0
  • globus-sdk 3.8.0
  • httplib2 0.14.0
  • idna 2.8
  • jmespath 0.10.0
  • keyring 18.0.1
  • language-selector 0.1
  • launchpadlib 1.10.13
  • lazr.restfulclient 0.14.2
  • lazr.uri 1.0.3
  • lockfile 0.12.2
  • louis 3.12.0
  • macaroonbakery 1.3.1
  • matplotlib 3.5.2
  • monotonic 1.5
  • netifaces 0.10.4
  • numpy 1.22.3
  • oauthlib 3.1.0
  • olefile 0.46
  • paramiko 2.6.0
  • pexpect 4.6.0
  • Pillow 7.0.0
  • pip 20.0.2
  • plotly 5.8.0
  • protobuf 3.6.1
  • pycairo 1.16.2
  • pycparser 2.21
  • pycups 1.9.73
  • PyGObject 3.36.0
  • PyJWT 2.3.0
  • pymacaroons 0.13.0
  • PyNaCl 1.3.0
  • pyRFC3339 1.1
  • PySimpleGUI 4.59.0
  • python-apt 2.0.0+ubuntu0.20.4.7
  • python-dateutil 2.7.3
  • python-debian 0.1.36ubuntu1
  • pytz 2019.3
  • pyxdg 0.26
  • PyYAML 5.3.1
  • reportlab 3.5.34
  • requests 2.22.0
  • requests-unixsocket 0.2.0
  • SecretStorage 2.3.1
  • setuptools 45.2.0
  • simplejson 3.16.0
  • six 1.14.0
  • ssh-import-id 5.10
  • systemd-python 234
  • typing-extensions 4.2.0
  • ubuntu-advantage-tools 27.7
  • ubuntu-drivers-common 0.0.0
  • ufw 0.36
  • unattended-upgrades 0.1
  • urllib3 1.25.8
  • usb-creator 0.3.7
  • wadllib 1.3.3
  • wheel 0.34.2
  • xkit 0.0.0

Uninterruptible Power Supply

Warning Warning :   shutdown protection not tested


A uninterruptible Power Supply (UPS) is used for power the data server, the Archive Data Server, and the Ethernet switches on the rack.

a program for monitoring the UPS is installed from https://www.cyberpowersystems.com/product/software/power-panel-business/powerpanel-business-4-8-1-linux/

The program provide a web interface. 

pauli.physics.fsu.edu:3052

for account and password, see elog

System monitoring

I use Prometheus + node_exporter. Same in the FSUNUC setup.

Others

Contact

Retrieved from "https://fsunuc.physics.fsu.edu/wiki/index.php?title=Data_Server&oldid=2472"