Requirements
============

- GNU Make
- Python (>= 3.3)
- Docker
- Docker Compose
- sudo
- cgroup v2: disabled
- RAM : 4GB or more
- The host OS is not used for other purposes

Disable cgroup v2
=================

- Is enabled cgroup v2 ?

  $ mount | grep "cgroup2 on /sys/fs/cgroup type"; echo $?
  0

  Zero means "cgroup v2 is enabled".

- Disable cgroup v2 when cgroup v2 is enabled.

  add the following parameters to GRUB_CMDLINE_LINUX_DEFAULT
  of /etc/default/grub :
  - systemd.unified_cgroup_hierarchy=false
  - systemd.legacy_systemd_cgroup_controller=false

  Example:
    GRUB_CMDLINE_LINUX_DEFAULT="quiet systemd.unified_cgroup_hierarchy=false systemd.legacy_systemd_cgroup_controller=false"

- update and reboot

  $ sudo update-grub
  $ sudo reboot

- Is enabled cgroup v2 ?

  $ mount | grep "cgroup2 on /sys/fs/cgroup type"; echo $?
  1

Quick start on Ubuntu 18.04 host
================================

1. Install Docker.
   See https://docs.docker.com/engine/install/ubuntu/ for details.

  $ sudo apt-get update
  $ sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common
  $ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
  $ sudo add-apt-repository \
    "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
    $(lsb_release -cs) stable"
  $ sudo apt-get update
  $ sudo apt-get install docker-ce docker-ce-cli containerd.io

2. Install Docker Compose.

  (Compose v2)
  See https://docs.docker.com/compose/cli-command/
      https://github.com/docker/compose
      for details.

  $ sudo mkdir -p /usr/local/libexec/docker/cli-plugins
  $ curl -SL https://github.com/docker/compose/releases/download/v2.0.1/docker-compose-linux-x86_64 -o /usr/local/libexec/docker/cli-plugins/docker-compose
  $ sudo chmod +x /usr/local/libexec/docker/cli-plugins/docker-compose

  (Compose v1)
  See https://docs.docker.com/compose/install/ for details.

  # specify the appropriate version.
  # cf. https://github.com/docker/compose/releases/latest
  $ VER=1.29.2
  $ curl -L \
    "https://github.com/docker/compose/releases/download/${VER}/docker-compose-$(uname -s)-$(uname -m)" \
    -o docker-compose \
  && sudo mv docker-compose /usr/local/bin/docker-compose
  $ sudo chmod +x /usr/local/bin/docker-compose

3. Install other packages.

  $ sudo apt install make git

4. Checkout gfarm2fs.

$ cd path/to/gfarm-src-top
$ git clone git@github.com:oss-tsukuba/gfarm2fs.git

5. Use Docker

  $ cd docker/dev
  $ editor config.mk
  # override GFDOCKER_IP_VERSION, GFDOCKER_SUBNET, and GFDOCKER_START_HOST_ADDR
  # from config-default.mk
  $ cd dist/centos7/src  # when using source package on CentOS 7
  # note: execute 'sudo' in Makefile
  $ make help  # display usage.
  $ make reborn  # up the containers.
  $ make shell  # enter the container.
  $ make regress  # execute regress.
  $ make down  # down the containers.

Keep keys
=========

1. to keep ssh host keys only for client1 container

  $ cd dist/centos7/src  # when using source package on CentOS 7
  $ make shell
  ### in container
  user1$ sudo ./SAVE_SSH_HOST_KEYS.sh

  Note: These keys are used next time in all distributions.

2. to add your ssh public keys to authorized keys only for user*@client1

  $ cd docker/dev
  ### add your public keys
  ### (Example)
  $ cat ~/.ssh/id_rsa.pub >> mnt/ssh_authorized_keys

  $ cd dist/centos7/src  # when using source package on CentOS 7
  $ make reborn

  ### use "ssh" instead of "make shell"
  $ ssh user2@192.168.224.11  # IP address of client1 container

Build test on all distributions
===============================

  $ cd docker/dev
  $ ./TEST_BUILD_ALL.sh

Note
====

- Do not edit .shadow.config.mk, docker-compose.yml, and common/rc.gfservice.

Debug
=====

* to dump core file in container

  (host_os)$ echo core.%p | sudo tee /proc/sys/kernel/core_pattern


Gfarm-S3-MinIO
==============

1. preparation: clone gfarm-s3-minio-web and gfarm-s3-minio

  $ mkdir -p docker/dev/mnt/work
  $ cd docker/dev/mnt/work
  $ git clone git@github.com:oss-tsukuba/gfarm-s3-minio-web.git
  $ cd gfarm-s3-minio-web
  $ git checkout develop
  $ cd ..
  $ git clone git@github.com:oss-tsukuba/gfarm-s3-minio.git
  $ cd gfarm-s3-minio
  $ git checkout gfarm

2. to install gfarm-s3 into running client1 container

  $ cd (docker/dev dir)
  $ cd dist/centos7/src
  $ make s3setup

3. to access

  (WebUI from Web Browser)
    http://{IP address of host OS}:18080/
      (GFDOCKER_HOSTPORT_S3_HTTP = 18080)
      or
    https://{IP address of host OS}:18443/
      (GFDOCKER_HOSTPORT_S3_HTTPS = 18443)

    username: user1
    password: <gfarm-s3-sharedsecret-password command output>

  - push "Start" button.
  - copy "Access Key ID" to clipboard later.
  - copy "Secret Access Key" to clipboard later.

  (S3 client from client1)
  $ make shell
  $ aws configure
  ### set "Access Key ID" and "Secret Access Key"
  $ aws s3 --endpoint-url http://{IP address of host OS}:18080 \
    mb s3://abcde

  $ mc ...
  $ s3fs ...
  $ goofys ...

4. to run s3 unit test

  $ cd docker/dev/dist/centos7/src
  $ make s3setup
  $ make s3test

5. update gfarm-s3-minio or gfarm-s3-minio-web

  $ cd docker/dev/mnt/work/
  ### edit or update gfarm-s3-minio and gfarm-s3-minio-web
  $ cd docker/dev/dist/centos7/src
  $ make s3update


Nextcloud using Gfarm as primary storage
========================================

1. preparation: clone nextcloud-gfarm

  $ mkdir -p docker/dev/mnt/work
  $ cd docker/dev/mnt/work
  $ git clone git@github.com:oss-tsukuba/nextcloud-gfarm.git
  $ cd nextcloud-gfarm
  $ git checkout master

2. to setup nextcloud-gfarm

  edit configurations:
  docker/dev/common/nextcloud/ :
  - docker-compose.nextcloud-gfarm.override.yml
    - ports
  - nextcloud.env
    - NEXTCLOUD_TRUSTED_DOMAINS

  $ cd docker/dev/dist/centos7/src
  $ make reborn
  $ make nextcloud-reborn

3. to restart

  use files on persistent docker volumes.
  (backup files are not used.)

  $ make nextcloud-restart

4. to show logs

  $ make nextcloud-logs-less
  $ make nextcloud-logs-f
  $ make nextcloud-logs

5. to destroy nextcloud-gfarm container

  (to leave Nextcloud persistent volumes)
  $ make nextcloud-rm

  (to clear Nextcloud persistent volumes)
  $ make nextcloud-rm-data

  (to destroy all Gfarm containers and Nextcloud persistent volumes)
  $ make down-with-nextcloud

6. to access Nextcloud WebUI

  (from Web Browser)
    http://{IP address of host OS}:28080/
    user: admin
    password: (from docker/dev/common/nextcloud/nextcloud_admin_password)

7. to backup into Gfarm

  executed periodically by crontab in Nextcloud container.

  (manually)
  $ make nextcloud-shell
  $ /backup.sh

  ### backup files are saved in gfarm:/home/user1/nextcloud-backup/

8. to restore from Gfarm

  (assume persistent volumes disappears)
  ### restoring from gfarm:/home/user1/nextcloud-backup automatically.

  $ make nextcloud-reborn
  ### Gfarm containers are not recreted.

9. upgrade Nextcloud version

  edit docker-compose.nextcloud-gfarm.override.yml
  # ex. (before ...) NEXTCLOUD_VERSION: 21
  #     (after  ...) NEXTCLOUD_VERSION: 22
  ## NOTE: It is only possible to upgrade one major version at a time.
  ### confirmed history:
  ### 20 -> 21
  ### 21 -> 22

  (not clear persistent volumes)
  $ make nextcloud-recreate

  (clear persistent volumes and restore from backup files on Gfarm)
  $ make nextcloud-reborn


Gfarm GridFTP DSI
=================

1. preparation: clone gfarm-gridftp-dsi

  $ mkdir -p docker/dev/mnt/work
  $ cd docker/dev/mnt/work
  $ git clone git@github.com:oss-tsukuba/gfarm-gridftp-dsi.git
  $ cd gfarm-gridftp-dsi
  $ git checkout <<branch name>>
  ### edit gfarm-gridftp-dsi

2. to install gfarm-gridftp-dsi into running container

  $ cd docker/dev/dist/centos7/src
  $ make gridftp-setup

3. to run unit test

  $ cd docker/dev/dist/centos7/src
  $ make gridftp-test
