RoE | Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Information ======

  * GNU MediaGoblin ((http://mediagoblin.org))
  * [[Information:Realm of Espionage]]
  * https://media.realmofespionage.xyz

===== Prerequisites =====

  * [[distros:ubuntu_server | Ubuntu Server]]
  * [[servers:nginx_php_php-fpm | nginx]]
  * [[servers:nginx:lets_encrypt | Let's Encrypt]]

===== Notes =====

  * AudioLab isn't compatible with Python 3: https://github.com/cournape/audiolab/issues/24
  * As of 2018/06/03, these notes support a picture-only instance ((no video, no audio))

====== Create Group and User ======

****

  sudo groupadd --force 'mediagoblin' && sudo useradd --comment 'GNU MediaGoblin system account' --home-dir '/var/lib/mediagoblin' --gid 'www-data' --create-home --system 'mediagoblin' && sudo usermod --append --groups 'mediagoblin' 'mediagoblin' && sync

====== Dependencies ======

****

  * opensuse

   npm nodejs6 automake python2-numpy python2-scipy python2-SQLAlchemy

  * deb MG site

  sudo apt-get install nodejs-legacy

  * Old notes

  sudo apt install gstreamer1.0-libav gstreamer1.0-plugins-good python-gst-1.0 gir1.2-gst-plugins-base-1.0 python-gi gstreamer1.0-tools python-numpy python-scipy libsndfile1-dev libasound2-dev python-sqlalchemy

  * TODO and eventually to be used

  sudo apt install python3 python3-dev python3-lxml python3-pil python3-virtualenv npm nodejs automake rabbitmq-server uwsgi uwsgi-plugin-python3

====== Download Source ======

****

  sudo git clone --branch 'master' --depth '1' --recurse-submodules 'http://git.savannah.gnu.org/r/mediagoblin.git' '/srv/www/media' && sudo chown -R 'mediagoblin':'nginx' '/srv/www/media' && sync

====== nginx + uWSGI Configuration ======

===== uWSGI Vassal =====

  sudo -e '/etc/uwsgi/vassals/media.ini' && sudo systemctl enable 'uwsgi' --now

<code>
[uwsgi]
master = true

enable-threads = true

plugin = python

uid = mediagoblin
gid = mediagoblin

socket = /run/media.sock
chown-socket = nginx:nginx

chdir = /srv/www/media
home = /srv/www/media
ini-paste = /srv/www/media/paste_local.ini
virtualenv = /srv/www/media</code>

===== uWSGI + nginx =====

  sudo -e '/etc/nginx/snippets.d/media.conf'

<code>
location / {
    include uwsgi_params;
    uwsgi_param SCRIPT_NAME "";
    uwsgi_pass unix:/run/media.sock;
}</code>

===== Server Block ======

  sudo -e '/etc/nginx/vhosts.d/media.conf' && sudo systemctl reload 'nginx'

<code>
server {
    listen '443' 'ssl' 'http2';
    server_name 'media.realmofespionage.xyz';
    client_max_body_size 10240M;

    include '/etc/nginx/snippets.d/media.conf';
    include '/etc/nginx/snippets.d/restrictions.conf';

    #access_log /var/log/nginx/media-access.log;
    #error_log /var/log/nginx/media-error.log;

    location /mgoblin_static/ {
        alias /srv/www/media/mediagoblin/static/;
    }

    location /mgoblin_media/ {
        alias /srv/www/media/user_dev/media/public/;
    }

    location /theme_static/ {
        alias /srv/www/media/user_dev/theme_static/;
    }

    location /plugin_static/ {
        alias /srv/www/media/user_dev/plugin_static/;
    }
}</code>

====== Install ======

===== Switch User =====

****

  sudo su 'mediagoblin' -s '/bin/bash'

===== Initialize Environment =====

****

  cd '/srv/www/media' && '/srv/www/media/bootstrap.sh' && '/srv/www/media/configure' && make || make

==== Create Media Directory ====

****

  mkdir -p '/srv/www/media/user_dev' && chmod 750 '/srv/www/media/user_dev' && sync

====== Settings ======

===== Change User =====

****

  sudo su 'mediagoblin' -s '/bin/bash'

===== Copy Configs =====

****

  cp -av '/srv/www/media/mediagoblin.ini' '/srv/www/media/mediagoblin_local.ini' && cp -av '/srv/www/media/paste.ini' '/srv/www/media/paste_local.ini'

===== GNU MediaGoblin =====

  nano '/srv/www/media/mediagoblin_local.ini'

<code>
[mediagoblin]
html_title = RoE | Media
allow_comments = false

allow_registration = false
allow_reporting = false</code>

===== Initialize Plugins and Database =====

****

  cd '/srv/www/media' && '/srv/www/media/bin/gmg' dbupdate && '/srv/www/media/bin/gmg' assetlink && sync

===== Create Account =====

****

  cd '/srv/www/media' && '/srv/www/media/bin/gmg' adduser --username 'Espionage724' && '/srv/www/media/bin/gmg' makeadmin 'Espionage724'

====== RabbitMQ ======

===== Config =====

  * Fixes an error ((''ERROR: epmd error for host Kraityn: address (cannot connect to host/port)''))
  * Before running this, test if it "just works" by running ''sudo rabbitmq-server'' first

  echo 'NODENAME=rabbit@localhost' | sudo tee --append '/etc/rabbitmq/rabbitmq-env.conf' > '/dev/null' && sudo systemctl enable 'rabbitmq-server' --now

====== Services ======

===== Celery =====

==== Service ====

  sudo -e '/etc/systemd/system/media-c.service' && sudo systemctl daemon-reload && sudo systemctl enable 'media-c' --now && sudo systemctl status 'media-c' -l

<code>
[Unit]
Description=GNU MediaGoblin Celery Task Queue Service
After=network-online.target
Wants=network-online.target

[Service]
User=mediagoblin
Group=mediagoblin
Type=simple
Environment=MEDIAGOBLIN_CONFIG=/var/www/media/mediagoblin_local.ini
Environment=CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery
WorkingDirectory=/var/www/media
ExecStart='/var/www/media/bin/celery' worker
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target</code>

===== Updater =====

==== Service ====

  sudo -e '/etc/systemd/system/media-up.service'

<code>
[Service]
User=mediagoblin
Group=mediagoblin
Type=oneshot
WorkingDirectory=/var/www/media
ExecStart='/usr/bin/git' -C '/var/www/media' pull origin 'master'
ExecStart='/usr/bin/git' -C '/var/www/media' submodule update
ExecStart='/var/www/media/bin/python2.7' '/var/www/media/setup.py' develop --upgrade
ExecStart='/var/www/media/bin/gmg' dbupdate
ExecStartPost='/bin/sync'</code>

==== Timer ====

  sudo -e '/etc/systemd/system/media-up.timer' && sudo systemctl daemon-reload && sudo systemctl enable 'media-up.timer' --now && sudo systemctl start 'media-up' && sudo systemctl status 'media-up' -l

<code>
[Unit]
Description=GNU MediaGoblin and Submodules Git Updater
After=network-online.target
Wants=network-online.target

[Timer]
OnCalendar=*-*-* 05:00:00
Persistent=true

[Install]
WantedBy=timers.target</code>

===== Restarter =====

==== Service ====

  * This script restarts both ''uwsgi'' and ''rabbitmq-server'', so if any other services in the future depend on either of these services being up, fix this in order to not touch the respective service

  sudo -e '/etc/systemd/system/media-re.service'

<code>
[Service]
Type=oneshot
ExecStartPre='/bin/sync'
ExecStart='/bin/systemctl' stop 'uwsgi'
ExecStart='/bin/systemctl' stop 'rabbitmq-server'
ExecStart='/bin/systemctl' stop 'media-c'
ExecStartPost='/bin/sync'
ExecStartPost='/bin/systemctl' start 'uwsgi'
ExecStartPost='/bin/systemctl' start 'rabbitmq-server'
ExecStartPost='/bin/systemctl' start 'media-c'</code>

==== Timer ====

  sudo -e '/etc/systemd/system/media-re.timer' && sudo systemctl daemon-reload && sudo systemctl enable 'media-re.timer' --now && sudo systemctl start 'media-re' && sudo systemctl status 'media-re' -l

<code>
[Unit]
Description=GNU MediaGoblin, uWSGI, and RabbitMQ Service Restarter
After=network-online.target
Wants=network-online.target

[Timer]
OnCalendar=*-*-* 05:30:00
Persistent=true

[Install]
WantedBy=timers.target</code>

===== Backup =====

==== User Directory ====

=== Service ===

  mkdir -p ~/'backups' && sudo -e '/etc/systemd/system/media-ufb.service' && sudo sed -i 's/CHANGEME/'$USER'/g' '/etc/systemd/system/media-ufb.service'

<code>
[Service]
Type=oneshot
WorkingDirectory=/var/lib
ExecStart='/bin/bash' -c '"/bin/tar" -cvzf "/home/CHANGEME/backups/mediagoblin-user-files-auto-"$$(date +%%Y-%%m-%%d)".tar.gz" "mediagoblin"'
ExecStartPost='/bin/sync'</code>

=== Timer ===

  * This happens monthly

  sudo -e '/etc/systemd/system/media-ufb.timer' && sudo systemctl daemon-reload && sudo systemctl enable 'media-ufb.timer' --now && sudo systemctl start 'media-ufb' && sudo systemctl status 'media-ufb' -l

<code>
[Unit]
Description=GNU MediaGoblin User Files Backup

[Timer]
OnCalendar=monthly
Persistent=true

[Install]
WantedBy=timers.target</code>

==== Web Folder ====

=== Service ===

  mkdir -p ~/'backups' && sudo -e '/etc/systemd/system/media-wfb.service' && sudo sed -i 's/CHANGEME/'$USER'/g' '/etc/systemd/system/media-wfb.service'

<code>
[Service]
Type=oneshot
WorkingDirectory=/var/www
ExecStart='/bin/bash' -c '"/bin/tar" -cvzf "/home/CHANGEME/backups/mediagoblin-web-files-auto-"$$(date +%%Y-%%m-%%d)".tar.gz" "media"'
ExecStartPost='/bin/sync''</code>

=== Timer ===

  * This happens monthly since the files folder is large

  sudo -e '/etc/systemd/system/media-wfb.timer' && sudo systemctl daemon-reload && sudo systemctl enable 'media-wfb.timer' --now && sudo systemctl start 'media-wfb' && sudo systemctl status 'media-wfb' -l

<code>
[Unit]
Description=GNU MediaGoblin Web Files Backup

[Timer]
OnCalendar=monthly
Persistent=true

[Install]
WantedBy=timers.target</code>

====== Backup ======

  * Create backup archive on server and transfer to client computer

===== Server =====

==== Stop Services ====

****

  sudo systemctl stop nginx uwsgi media-c

==== Backup User Folder ====

****

  cd '/var/lib' && sudo tar -cvzf ~/'mediagoblin-user-files-manual-'$(date +%Y-%m-%d)'.tar.gz' 'mediagoblin' && cd ~ && sync

==== Backup Web Folder ====

****

  cd '/var/www' && sudo tar -cvzf ~/'mediagoblin-web-files-manual-'$(date +%Y-%m-%d)'.tar.gz' 'media' && cd ~ && sync

==== Start Services ====

****

  sudo systemctl start nginx uwsgi media-c

===== Client =====

==== Transfer Files To Client ====

****

  scp espionage724@192.168.1.153:~/'mediagoblin-user-files-'*'.tar.gz' ~/'mediagoblin-web-files-'*'.tar.gz' ~/'Downloads' && sync

====== Restore ======

===== Client =====

==== Transfer Files To Server ====

****

  scp ~/'Downloads/mediagoblin-user-files-'*'.tar.gz' ~/'Downloads/mediagoblin-web-files-'*'.tar.gz' espionage724@192.168.1.153:~

==== Remove Files ====

****

  rm -f ~/'Downloads/mediagoblin-user-files-'*'.tar.gz' ~/'Downloads/mediagoblin-web-files-'*'.tar.gz' && sync

===== Server =====

==== Stop Services ====

****

  sudo systemctl stop nginx uwsgi media-c

==== Remove Previous Folders ====

****

  sudo rm -Rf '/var/lib/mediagoblin' '/var/www/media'

==== Restore GNU MediaGoblin User Folder ====

****

  cd '/var/lib' && sudo tar -xvzf ~/'mediagoblin-user-files-'*'.tar.gz' 'mediagoblin' && sudo chown -R 'mediagoblin':'www-data' '/var/lib/mediagoblin' && cd ~ && sync

==== Restore GNU MediaGoblin Web Folder ====

****

  cd '/var/www' && sudo tar -xvzf ~/'mediagoblin-web-files-'*'.tar.gz' 'media' && sudo chown -R 'mediagoblin':'www-data' '/var/www/media' && cd ~ && sync

==== Start Services ====

****

  sudo systemctl start nginx uwsgi media-c

==== Remove Backups ====

  * Verify that GNU MediaGoblin works before running

  rm ~/'mediagoblin-user-files-'*'.tar.gz' ~/'mediagoblin-web-files-'*'.tar.gz' && sync