How to Install and Configure GlusterFS on Ubuntu 22.04
This tutorial exists for these OS versions
- Ubuntu 22.04 (Jammy Jellyfish)
- Ubuntu 18.04 (Bionic Beaver)
- Ubuntu 18.04 (Bionic Beaver)
- Ubuntu 12.10 (Quantal Quetzal)
- Ubuntu 12.04 LTS (Precise Pangolin)
- Ubuntu 11.10 (Oneiric Ocelot)
On this page
GlusterFS is a free, open-source and software-defined distributed storage developed by Gluster Inc, and then by Redhat. It provides interfaces for object, block, and file storage and is used for high data-intensive workloads such as cloud storage, CDN, and media streaming. It can scale out in a building-block fashion to store multiple petabytes of data. It supports multiple protocols such as NFS, SMB, CIFS, HTTP, and FTP. It is perfectly suited for various use cases that require the handling of large amounts of data.
In this tutorial, we will learn how to set up Replicated GlusterFS Volume on Ubuntu 22.04.
Requirements
- Two servers running Ubuntu 22.04 with 2 GB external Hardisk on each.
- A static IP address 192.168.10.10 is configured on node1 and 192.168.10.11 is configured on node2.
- A root password is configured on both servers.
Getting Started
Before starting, Updating all system packages on both servers is recommended. You can update them by running the following command on both servers.
apt-get update -y
apt-get upgrade -y
Once all the system packages are updated, restart them to apply the changes.
Next, you will also need to edit the /etc/hosts file on both servers and set up hostname resolution so each server can communicate with the other by hostname. You can do it with the following command:
nano /etc/hosts
Add the following lines:
192.168.10.10 node1 192.168.10.11 node2
Save and close the file when you are finished.
Install GlusterFS Server
By default, GlusterFS is available in the Ubuntu 22.04 default repository. You can install it on both servers using the following command.
apt-get install glusterfs-server -y
Once the GlusterFS package has been installed, start the GlusterFS service and enable it to start on boot with the following command:
systemctl start glusterd
systemctl enable glusterd
You can also check the status of the GlusterFS service with the following command:
systemctl status glusterd
You should see the following output:
? glusterd.service - GlusterFS, a clustered file-system server
Loaded: loaded (/lib/systemd/system/glusterd.service; disabled; vendor preset: enabled)
Active: active (running) since Sat 2022-11-05 04:10:13 UTC; 7s ago
Docs: man:glusterd(8)
Process: 92014 ExecStart=/usr/sbin/glusterd -p /var/run/glusterd.pid --log-level $LOG_LEVEL $GLUSTERD_OPTIONS (code=exited, status=0/SUCC>
Main PID: 92015 (glusterd)
Tasks: 9 (limit: 4579)
Memory: 7.1M
CPU: 2.260s
CGroup: /system.slice/glusterd.service
??92015 /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO
Nov 05 04:10:11 ubuntu2204 systemd[1]: Starting GlusterFS, a clustered file-system server...
Nov 05 04:10:13 ubuntu2204 systemd[1]: Started GlusterFS, a clustered file-system server.
Once you are finished, you can proceed to the next step.
Create GlusterFS Storage
To create a GlusterFS storage, you will need an external hard disk on both servers. You will also need to create a partition on an external harddisk(/dev/sdb) on both servers.
To create a new partition on /dev/sdb, run the following command:
fdisk /dev/sdb
You should see the following output:
Welcome to fdisk (util-linux 2.31.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0x06091fe8.
Command (m for help): n
Partition type
p primary (0 primary, 0 extended, 4 free)
e extended (container for logical partitions)
Select (default p): p
Partition number (1-4, default 1):
First sector (2048-4194303, default 2048):
Last sector, +sectors or +size{K,M,G,T,P} (2048-4194303, default 4194303):
Created a new partition 1 of type 'Linux' and of size 2 GiB.
Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
Next, format the partition with the following command:
mkfs.xfs /dev/sdb1
You should see the following output:
meta-data=/dev/sdb1 isize=512 agcount=4, agsize=131008 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0
data = bsize=4096 blocks=524032, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
Next, create a directory for GlusterFS storage on both servers with the following command:
mkdir /glustervolume
Next, mount the disk /dev/sdb1 on a directory /glustervolume.
mount.xfs /dev/sdb1 /glustervolume
Next, edit the /etc/fstab file and make the mount persistent across reboot:
nano /etc/fstab
Add the following line:
/dev/sdb1 /glustervolume xfs defaults 0 0
Save and close the file then run the following command to apply the changes:
mount -a
You can now check the mounted directory with the following command:
df -h
You should see the following output:
Filesystem Size Used Avail Use% Mounted on tmpfs 393M 1.1M 392M 1% /run /dev/sda2 79G 8.6G 67G 12% / tmpfs 2.0G 0 2.0G 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 393M 4.0K 393M 1% /run/user/0 /dev/sdb1 2.0G 35M 2.0G 2% /glustervolume
Configure GlusterFS Volume
First, you will need to create a trusted storage pool by adding node2 to node1. To do so, run the following command on node1:
gluster peer probe node2
You can verify the added storage pool with the following command:
gluster pool list
You should get the following output:
UUID Hostname State 52245177-ce2e-4217-8da5-2ee7a33dcf46 node2 Connected b6da6e83-b1ea-46de-8904-1c5345b1c229 localhost Connected
Next, create a brick directory on both nodes with the following command:
mkdir /glustervolume/vol1
Next, create a volume named vol1 with two replicas:
gluster volume create vol1 replica 2 node1:/glustervolume/vol1 node2:/glustervolume/vol1
Next, start the volume using the following command:
gluster volume start vol1
You can check the status of the created volume with the following command:
gluster volume status
You should get the following output:
Status of volume: vol1 Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------ Brick node1:/glustervolume/vol1 49152 0 Y 7198 Brick node2:/glustervolume/vol1 49152 0 Y 13214 Self-heal Daemon on localhost N/A N/A Y 6121 Self-heal Daemon on node2 N/A N/A Y 12768 Task Status of Volume vol1 ------------------------------------------------------------------------------ There are no active volume tasks
You can now check the information about created volume using the following command:
gluster volume info vol1
You should get the following output:
Volume Name: vol1 Type: Replicate Volume ID: a1023sd6d-81rt-12a2-bc4e-a4gd156c5331 Status: Started Snapshot Count: 0 Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: node1:/glustervolume/vol1 Brick2: node2:/glustervolume/vol1 Options Reconfigured: transport.address-family: inet nfs.disable: on performance.client-io-threads: off
Next, mount the created volume on /opt directory on both nodes by editing /etc/fstab file so your volume is available after the server boots.
nano /etc/fstab
Add the following line:
localhost:/vol1 /opt glusterfs defaults,_netdev 0 0
Save and close the file when you are finished. Then, run the following command to apply the changes:
mount -a
Test Replication
Your GlusterFS server is now installed and configured. Now, it's time to test replication between both nodes.
First, create two files on node1:
touch /opt/test1 /opt/test2
Now, go to node2 and run the following command:
ls -l /opt/
You should see the same files that you have created on node1:
total 0 -rw-r--r-- 1 root root 0 Nov 7 13:42 file1 -rw-r--r-- 1 root root 0 Nov 7 13:42 file2
Now, stop the GlusterFS service on node2 with the following command:
service glusterd stop
Next, go to the node1 and check the peer status:
gluster peer status
You should see that node2 has been disconnected:
Number of Peers: 1 Hostname: node2 Uuid: a134f146-1220-3avd-d3f6-1c2554a87sa1 State: Peer in Cluster (Disconnected)
You can also check the volume status with the following command:
gluster volume status
You should see the following output:
Status of volume: vol1 Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------ Brick node1:/glustervolume/vol1 49152 0 Y 7987 Self-heal Daemon on localhost N/A N/A Y 8010 Task Status of Volume vol1 ------------------------------------------------------------------------------ There are no active volume tasks
Next, create two other files on node1:
touch /opt/test3 /opt/test4
Now, go to the node2 and start the GlusterFS service with the following command:
service glusterd start
Now, check the /opt directory on node2:
ls -l /opt/
You should see that all files from node1 have been replicated and are now available.
total 0 -rw-r--r-- 1 root root 0 Nov 7 13:42 test1 -rw-r--r-- 1 root root 0 Nov 7 13:42 test2 -rw-r--r-- 1 root root 0 Nov 7 13:44 test3 -rw-r--r-- 1 root root 0 Nov 7 13:44 test4
Conclusion
Congratulations! you have successfully installed and configured a two-node Replicated Glusterfs server on Ubuntu 22.04 server. For more information, read the GlusterFS documentation page. Feel free to ask me if you have any questions.
