Archive Data Server: Difference between revisions

From FSU Fox's Lab Wiki
Jump to navigation Jump to search
No edit summary
No edit summary
 
(41 intermediate revisions by the same user not shown)
Line 1: Line 1:
The archive data server is build upon an relic Dell R710 rack server, which was bought around 2010 and haven't been using for long time. It was a format [[ANASEN]] DAQ computer. The server has 8 TB of storage.  
The archive data server is built upon a relic Dell R710 rack server, which was bought around 2010 and hasn't been used for long time. It was a former [[ANASEN]] DAQ computer. The server has 8 TB of storage.  


Users home directories and data and code of finished experiments will be stored in the server.  
User's home directories and data and code of finished experiments will be stored on the server.
 
The directories are
 
{| class='wikitable'
! directory || purpose
|-
| /archiveData || for archive data
|-
| /backupHome || for backing up Home directory
|}
 
 
= Solution on installing Modern OS =
 
Modern OS, like Ubuntu 20, use UEFI boot that is different from BIOS boot.
 
This is an old machine booting from BIOS that read only the 1st 512kB for the master boot record and the 2nd 512kB for the boottrap.img. It provides UEFI boot manual (so that Ubuntu 20 USB boot drive can run, with BIOD update, BIOS mode can also boot Ubuntu 20 USB), but, the RAID controller does not support the UEFI (in fact, it clearly shows that the Virtual Disk is supported by BIOS), so when boosting the RAID array from UEFI, it switches back to BIOS mode. That's why an installed Ubuntu 20.04 cannot boot and return the grub command line. In fact, in the grub command line, it only sees 2 partitions, the (proc) and (hd0), which are both 512kB. This indicates that it is using BIOS mode to see the first two 512kB for the boot loader.
 
There is no way to bypass the RAID controller, as it is the only interface to connect hard disks to the motherboard.
 
The problem is the grub cannot read a partition larger than 100 GB. In order to cope with that, we need to set the partition manually
* 1 MB for BIOS_boot (master boot record), unformatted
* 10 GB for /boot , ext4
* 8 GB swap
* 500 MB for EFI System, FAT
* Rest /, ext4
 
Then Ubuntu 22.04 is able to installed and boot
 
=== an idea ===
#Install a PCIe card with NVMe M.2 SSD, and boot to the SSD?
#replace the CD-ROM with an SSD. The CD-ROM is using SATA cable to connect, power?


= Network =  
= Network =  


IP : 128.186.111.52
IP : 128.186.111.52
Alias : newton.physics.fsu.edu
Alias : newton.physics.fsu.edu


= Spec. =
= Spec. =
Dell R710 rack server (11th gen, around 2010)
Dell R710 rack server (11th gen, around 2010) The Hardware Manual is [[:File:R710 hardware manual.pdf| Here]]
 
CPU : Intel Xeon E5606 Quad-core @ 2.13 GHz x 2 = total 8 cores
 
RAM : 2 x 4 GB DDR3 RDIMM at 1067 MHz.
 
RAID controller : PERC 6/i, PCIe 1.0 x 8, Data Transfer rate 3 Gb/s (SAS SFF-8484 32Pin), max disk size = 2 TB
 
storage: 4 x 2 TB RAID 5 = 6 TB
 
OS : Ubuntu 22.04
 
== solution to increase the supported disk size ==
 
# a firmware upgrade to 4 TB (rumor? which firmware version at the present?)
#: https://www.dell.com/support/home/en-us/drivers/driversdetails?driverid=f96nr&oscode=naa&productcode=poweredge-r710
# replace the PERC 6/i RAID controller with H700 (~ $65 from Amazon), which supports 6 TB. The cable also needs to change
#: see this post https://www.reddit.com/r/sysadmin/comments/2pjuy3/perc_6i_controller_with_6tb_drives/
 
Some people suggest that it is related to the sector size. The hard disk in the past used 512 bytes for a sector size and the partition table support 2^32 bit, so 2 TB is the max disk size. From the transition to 512 bytes to 4 KB sector size, there is a 512 emulation (512e). But it supports a 4 ~ 6 TB disk, not more.
 
using fdisk -l
 
~>sudo fdisk -l /dev/sda
Disk /dev/sda: 5.46 TiB, 5999532441600 bytes, 11717836800 sectors
Disk model: PERC 6/i
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


OS : lubuntu 20.04
indicate that it is using 512 bytes sector size (physical).  


storage: 5 x 2 TB RAID 5 = 8 TB
For the old 2 TB disk in [[Web Server|fsunuc]],
Disk /dev/sdb: 1.84 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: ST2000VN004-2E41
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Max hard disk size supported : 2 TB
= Update BIOS =


= Update iDrac and BIOS =
The origin BIOS version is 3.3.0. It does not allow boots from UEFI USB disk in BIOS mode. After upgrading to 6.6.0 it is OK.


use downloads.dell.com
#prepare an empty USB stick
#have a windows PC
#download the RUFUS
#download the latest BIOS 6.6.0 https://dl.dell.com/FOLDER05012843M/1/R710-060600C.exe
#use the window PC, run RUFUS
##select FreeDOS
##Check the 2nd, 3rd and 4th check boxes
##make the bootable disk
#in Dell R710, F11 to boot manual (if UEFI, switch to BIOS)
#Select "Hard Drive C:"
##Select USB
#then it will go to DOS
#run the R710-060600C.exe
#follow the instruction

Latest revision as of 15:24, 25 June 2022

The archive data server is built upon a relic Dell R710 rack server, which was bought around 2010 and hasn't been used for long time. It was a former ANASEN DAQ computer. The server has 8 TB of storage.

User's home directories and data and code of finished experiments will be stored on the server.

The directories are

directory purpose
/archiveData for archive data
/backupHome for backing up Home directory


Solution on installing Modern OS

Modern OS, like Ubuntu 20, use UEFI boot that is different from BIOS boot.

This is an old machine booting from BIOS that read only the 1st 512kB for the master boot record and the 2nd 512kB for the boottrap.img. It provides UEFI boot manual (so that Ubuntu 20 USB boot drive can run, with BIOD update, BIOS mode can also boot Ubuntu 20 USB), but, the RAID controller does not support the UEFI (in fact, it clearly shows that the Virtual Disk is supported by BIOS), so when boosting the RAID array from UEFI, it switches back to BIOS mode. That's why an installed Ubuntu 20.04 cannot boot and return the grub command line. In fact, in the grub command line, it only sees 2 partitions, the (proc) and (hd0), which are both 512kB. This indicates that it is using BIOS mode to see the first two 512kB for the boot loader.

There is no way to bypass the RAID controller, as it is the only interface to connect hard disks to the motherboard.

The problem is the grub cannot read a partition larger than 100 GB. In order to cope with that, we need to set the partition manually

  • 1 MB for BIOS_boot (master boot record), unformatted
  • 10 GB for /boot , ext4
  • 8 GB swap
  • 500 MB for EFI System, FAT
  • Rest /, ext4

Then Ubuntu 22.04 is able to installed and boot

an idea

  1. Install a PCIe card with NVMe M.2 SSD, and boot to the SSD?
  2. replace the CD-ROM with an SSD. The CD-ROM is using SATA cable to connect, power?

Network

IP : 128.186.111.52

Alias : newton.physics.fsu.edu

Spec.

Dell R710 rack server (11th gen, around 2010) The Hardware Manual is Here

CPU : Intel Xeon E5606 Quad-core @ 2.13 GHz x 2 = total 8 cores

RAM : 2 x 4 GB DDR3 RDIMM at 1067 MHz.

RAID controller : PERC 6/i, PCIe 1.0 x 8, Data Transfer rate 3 Gb/s (SAS SFF-8484 32Pin), max disk size = 2 TB

storage: 4 x 2 TB RAID 5 = 6 TB

OS : Ubuntu 22.04

solution to increase the supported disk size

  1. a firmware upgrade to 4 TB (rumor? which firmware version at the present?)
    https://www.dell.com/support/home/en-us/drivers/driversdetails?driverid=f96nr&oscode=naa&productcode=poweredge-r710
  2. replace the PERC 6/i RAID controller with H700 (~ $65 from Amazon), which supports 6 TB. The cable also needs to change
    see this post https://www.reddit.com/r/sysadmin/comments/2pjuy3/perc_6i_controller_with_6tb_drives/

Some people suggest that it is related to the sector size. The hard disk in the past used 512 bytes for a sector size and the partition table support 2^32 bit, so 2 TB is the max disk size. From the transition to 512 bytes to 4 KB sector size, there is a 512 emulation (512e). But it supports a 4 ~ 6 TB disk, not more.

using fdisk -l 

~>sudo fdisk -l /dev/sda
Disk /dev/sda: 5.46 TiB, 5999532441600 bytes, 11717836800 sectors
Disk model: PERC 6/i
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

indicate that it is using 512 bytes sector size (physical).

For the old 2 TB disk in fsunuc, 

Disk /dev/sdb: 1.84 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: ST2000VN004-2E41
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Update BIOS

The origin BIOS version is 3.3.0. It does not allow boots from UEFI USB disk in BIOS mode. After upgrading to 6.6.0 it is OK.

  1. prepare an empty USB stick
  2. have a windows PC
  3. download the RUFUS
  4. download the latest BIOS 6.6.0 https://dl.dell.com/FOLDER05012843M/1/R710-060600C.exe
  5. use the window PC, run RUFUS
    1. select FreeDOS
    2. Check the 2nd, 3rd and 4th check boxes
    3. make the bootable disk
  6. in Dell R710, F11 to boot manual (if UEFI, switch to BIOS)
  7. Select "Hard Drive C:"
    1. Select USB
  8. then it will go to DOS
  9. run the R710-060600C.exe
  10. follow the instruction
Retrieved from "https://fsunuc.physics.fsu.edu/wiki/index.php?title=Archive_Data_Server&oldid=1117"