User Tools

Site Tools


servers:nginx:gnu_mediagoblin

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
servers:nginx:gnu_mediagoblin [2018/07/23 13:26]
Sean Rhone [Dependencies]
servers:nginx:gnu_mediagoblin [2018/09/18 13:31] (current)
Sean Rhone [Download Source]
Line 1: Line 1:
 +====== Information ======
  
 +  * GNU MediaGoblin ((http://mediagoblin.org))
 +  * [[Information:Realm of Espionage]]
 +  * https://media.realmofespionage.xyz
 +
 +===== Prerequisites =====
 +
 +  * [[distros:ubuntu_server | Ubuntu Server]]
 +  * [[servers:nginx_php_php-fpm | nginx]]
 +  * [[servers:nginx:lets_encrypt | Let's Encrypt]]
 +
 +===== Notes =====
 +
 +  * AudioLab isn't compatible with Python 3: https://github.com/cournape/audiolab/issues/24
 +  * As of 2018/06/03, these notes support a picture-only instance ((no video, no audio))
 +
 +====== Create Group and User ======
 +
 +****
 +
 +  sudo groupadd --force 'mediagoblin' && sudo useradd --comment 'GNU MediaGoblin system account' --home-dir '/var/lib/mediagoblin' --gid 'www-data' --create-home --system 'mediagoblin' && sudo usermod --append --groups 'mediagoblin' 'mediagoblin' && sync
 +
 +====== Dependencies ======
 +
 +****
 +
 +  * opensuse
 +
 +   npm nodejs6 automake python2-numpy python2-scipy python2-SQLAlchemy
 +
 +  * deb MG site
 +
 +  sudo apt-get install nodejs-legacy
 +
 +  * Old notes
 +
 +  sudo apt install gstreamer1.0-libav gstreamer1.0-plugins-good python-gst-1.0 gir1.2-gst-plugins-base-1.0 python-gi gstreamer1.0-tools python-numpy python-scipy libsndfile1-dev libasound2-dev python-sqlalchemy
 +
 +  * TODO and eventually to be used
 +
 +  sudo apt install python3 python3-dev python3-lxml python3-pil python3-virtualenv npm nodejs automake rabbitmq-server uwsgi uwsgi-plugin-python3
 +
 +====== Download Source ======
 +
 +****
 +
 +  sudo git clone --branch 'master' --depth '1' --recurse-submodules 'http://git.savannah.gnu.org/r/mediagoblin.git' '/srv/www/media' && sudo chown -R 'mediagoblin':'nginx' '/srv/www/media' && sync
 +
 +====== nginx + uWSGI Configuration ======
 +
 +===== uWSGI Vassal =====
 +
 +  sudo -e '/etc/uwsgi/vassals/media.ini' && sudo systemctl enable 'uwsgi' --now
 +
 +<code>
 +[uwsgi]
 +master = true
 +
 +enable-threads = true
 +
 +plugin = python
 +
 +uid = mediagoblin
 +gid = mediagoblin
 +
 +socket = /run/media.sock
 +chown-socket = nginx:nginx
 +
 +chdir = /srv/www/media
 +home = /srv/www/media
 +ini-paste = /srv/www/media/paste_local.ini
 +virtualenv = /srv/www/media</code>
 +
 +===== uWSGI + nginx =====
 +
 +  sudo -e '/etc/nginx/snippets.d/media.conf'
 +
 +<code>
 +location / {
 +    include uwsgi_params;
 +    uwsgi_param SCRIPT_NAME "";
 +    uwsgi_pass unix:/run/media.sock;
 +}</code>
 +
 +===== Server Block ======
 +
 +  sudo -e '/etc/nginx/vhosts.d/media.conf' && sudo systemctl reload 'nginx'
 +
 +<code>
 +server {
 +    listen '443' 'ssl' 'http2';
 +    server_name 'media.realmofespionage.xyz';
 +    client_max_body_size 10240M;
 +
 +    include '/etc/nginx/snippets.d/media.conf';
 +    include '/etc/nginx/snippets.d/restrictions.conf';
 +
 +    #access_log /var/log/nginx/media-access.log;
 +    #error_log /var/log/nginx/media-error.log;
 +
 +    location /mgoblin_static/ {
 +        alias /srv/www/media/mediagoblin/static/;
 +    }
 +
 +    location /mgoblin_media/ {
 +        alias /srv/www/media/user_dev/media/public/;
 +    }
 +
 +    location /theme_static/ {
 +        alias /srv/www/media/user_dev/theme_static/;
 +    }
 +
 +    location /plugin_static/ {
 +        alias /srv/www/media/user_dev/plugin_static/;
 +    }
 +}</code>
 +
 +====== Install ======
 +
 +===== Switch User =====
 +
 +****
 +
 +  sudo su 'mediagoblin' -s '/bin/bash'
 +
 +===== Initialize Environment =====
 +
 +****
 +
 +  cd '/srv/www/media' && '/srv/www/media/bootstrap.sh' && '/srv/www/media/configure' && make || make
 +
 +==== Create Media Directory ====
 +
 +****
 +
 +  mkdir -p '/srv/www/media/user_dev' && chmod 750 '/srv/www/media/user_dev' && sync
 +
 +====== Settings ======
 +
 +===== Change User =====
 +
 +****
 +
 +  sudo su 'mediagoblin' -s '/bin/bash'
 +
 +===== Copy Configs =====
 +
 +****
 +
 +  cp -av '/srv/www/media/mediagoblin.ini' '/srv/www/media/mediagoblin_local.ini' && cp -av '/srv/www/media/paste.ini' '/srv/www/media/paste_local.ini'
 +
 +===== GNU MediaGoblin =====
 +
 +  nano '/srv/www/media/mediagoblin_local.ini'
 +
 +<code>
 +[mediagoblin]
 +html_title = RoE | Media
 +allow_comments = false
 +
 +allow_registration = false
 +allow_reporting = false</code>
 +
 +===== Initialize Plugins and Database =====
 +
 +****
 +
 +  cd '/srv/www/media' && '/srv/www/media/bin/gmg' dbupdate && '/srv/www/media/bin/gmg' assetlink && sync
 +
 +===== Create Account =====
 +
 +****
 +
 +  cd '/srv/www/media' && '/srv/www/media/bin/gmg' adduser --username 'Espionage724' && '/srv/www/media/bin/gmg' makeadmin 'Espionage724'
 +
 +====== RabbitMQ ======
 +
 +===== Config =====
 +
 +  * Fixes an error ((''ERROR: epmd error for host Kraityn: address (cannot connect to host/port)''))
 +  * Before running this, test if it "just works" by running ''sudo rabbitmq-server'' first
 +
 +  echo 'NODENAME=rabbit@localhost' | sudo tee --append '/etc/rabbitmq/rabbitmq-env.conf' > '/dev/null' && sudo systemctl enable 'rabbitmq-server' --now
 +
 +====== Services ======
 +
 +===== Celery =====
 +
 +==== Service ====
 +
 +  sudo -e '/etc/systemd/system/media-c.service' && sudo systemctl daemon-reload && sudo systemctl enable 'media-c' --now && sudo systemctl status 'media-c' -l
 +
 +<code>
 +[Unit]
 +Description=GNU MediaGoblin Celery Task Queue Service
 +After=network-online.target
 +Wants=network-online.target
 +
 +[Service]
 +User=mediagoblin
 +Group=mediagoblin
 +Type=simple
 +Environment=MEDIAGOBLIN_CONFIG=/var/www/media/mediagoblin_local.ini
 +Environment=CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery
 +WorkingDirectory=/var/www/media
 +ExecStart='/var/www/media/bin/celery' worker
 +Restart=always
 +RestartSec=5
 +
 +[Install]
 +WantedBy=multi-user.target</code>
 +
 +===== Updater =====
 +
 +==== Service ====
 +
 +  sudo -e '/etc/systemd/system/media-up.service'
 +
 +<code>
 +[Service]
 +User=mediagoblin
 +Group=mediagoblin
 +Type=oneshot
 +WorkingDirectory=/var/www/media
 +ExecStart='/usr/bin/git' -C '/var/www/media' pull origin 'master'
 +ExecStart='/usr/bin/git' -C '/var/www/media' submodule update
 +ExecStart='/var/www/media/bin/python2.7' '/var/www/media/setup.py' develop --upgrade
 +ExecStart='/var/www/media/bin/gmg' dbupdate
 +ExecStartPost='/bin/sync'</code>
 +
 +==== Timer ====
 +
 +  sudo -e '/etc/systemd/system/media-up.timer' && sudo systemctl daemon-reload && sudo systemctl enable 'media-up.timer' --now && sudo systemctl start 'media-up' && sudo systemctl status 'media-up' -l
 +
 +<code>
 +[Unit]
 +Description=GNU MediaGoblin and Submodules Git Updater
 +After=network-online.target
 +Wants=network-online.target
 +
 +[Timer]
 +OnCalendar=*-*-* 05:00:00
 +Persistent=true
 +
 +[Install]
 +WantedBy=timers.target</code>
 +
 +===== Restarter =====
 +
 +==== Service ====
 +
 +  * This script restarts both ''uwsgi'' and ''rabbitmq-server'', so if any other services in the future depend on either of these services being up, fix this in order to not touch the respective service
 +
 +  sudo -e '/etc/systemd/system/media-re.service'
 +
 +<code>
 +[Service]
 +Type=oneshot
 +ExecStartPre='/bin/sync'
 +ExecStart='/bin/systemctl' stop 'uwsgi'
 +ExecStart='/bin/systemctl' stop 'rabbitmq-server'
 +ExecStart='/bin/systemctl' stop 'media-c'
 +ExecStartPost='/bin/sync'
 +ExecStartPost='/bin/systemctl' start 'uwsgi'
 +ExecStartPost='/bin/systemctl' start 'rabbitmq-server'
 +ExecStartPost='/bin/systemctl' start 'media-c'</code>
 +
 +==== Timer ====
 +
 +  sudo -e '/etc/systemd/system/media-re.timer' && sudo systemctl daemon-reload && sudo systemctl enable 'media-re.timer' --now && sudo systemctl start 'media-re' && sudo systemctl status 'media-re' -l
 +
 +<code>
 +[Unit]
 +Description=GNU MediaGoblin, uWSGI, and RabbitMQ Service Restarter
 +After=network-online.target
 +Wants=network-online.target
 +
 +[Timer]
 +OnCalendar=*-*-* 05:30:00
 +Persistent=true
 +
 +[Install]
 +WantedBy=timers.target</code>
 +
 +===== Backup =====
 +
 +==== User Directory ====
 +
 +=== Service ===
 +
 +  mkdir -p ~/'backups' && sudo -e '/etc/systemd/system/media-ufb.service' && sudo sed -i 's/CHANGEME/'$USER'/g' '/etc/systemd/system/media-ufb.service'
 +
 +<code>
 +[Service]
 +Type=oneshot
 +WorkingDirectory=/var/lib
 +ExecStart='/bin/bash' -c '"/bin/tar" -cvzf "/home/CHANGEME/backups/mediagoblin-user-files-auto-"$$(date +%%Y-%%m-%%d)".tar.gz" "mediagoblin"'
 +ExecStartPost='/bin/sync'</code>
 +
 +=== Timer ===
 +
 +  * This happens monthly
 +
 +  sudo -e '/etc/systemd/system/media-ufb.timer' && sudo systemctl daemon-reload && sudo systemctl enable 'media-ufb.timer' --now && sudo systemctl start 'media-ufb' && sudo systemctl status 'media-ufb' -l
 +
 +<code>
 +[Unit]
 +Description=GNU MediaGoblin User Files Backup
 +
 +[Timer]
 +OnCalendar=monthly
 +Persistent=true
 +
 +[Install]
 +WantedBy=timers.target</code>
 +
 +==== Web Folder ====
 +
 +=== Service ===
 +
 +  mkdir -p ~/'backups' && sudo -e '/etc/systemd/system/media-wfb.service' && sudo sed -i 's/CHANGEME/'$USER'/g' '/etc/systemd/system/media-wfb.service'
 +
 +<code>
 +[Service]
 +Type=oneshot
 +WorkingDirectory=/var/www
 +ExecStart='/bin/bash' -c '"/bin/tar" -cvzf "/home/CHANGEME/backups/mediagoblin-web-files-auto-"$$(date +%%Y-%%m-%%d)".tar.gz" "media"'
 +ExecStartPost='/bin/sync''</code>
 +
 +=== Timer ===
 +
 +  * This happens monthly since the files folder is large
 +
 +  sudo -e '/etc/systemd/system/media-wfb.timer' && sudo systemctl daemon-reload && sudo systemctl enable 'media-wfb.timer' --now && sudo systemctl start 'media-wfb' && sudo systemctl status 'media-wfb' -l
 +
 +<code>
 +[Unit]
 +Description=GNU MediaGoblin Web Files Backup
 +
 +[Timer]
 +OnCalendar=monthly
 +Persistent=true
 +
 +[Install]
 +WantedBy=timers.target</code>
 +
 +====== Backup ======
 +
 +  * Create backup archive on server and transfer to client computer
 +
 +===== Server =====
 +
 +==== Stop Services ====
 +
 +****
 +
 +  sudo systemctl stop nginx uwsgi media-c
 +
 +==== Backup User Folder ====
 +
 +****
 +
 +  cd '/var/lib' && sudo tar -cvzf ~/'mediagoblin-user-files-manual-'$(date +%Y-%m-%d)'.tar.gz' 'mediagoblin' && cd ~ && sync
 +
 +==== Backup Web Folder ====
 +
 +****
 +
 +  cd '/var/www' && sudo tar -cvzf ~/'mediagoblin-web-files-manual-'$(date +%Y-%m-%d)'.tar.gz' 'media' && cd ~ && sync
 +
 +==== Start Services ====
 +
 +****
 +
 +  sudo systemctl start nginx uwsgi media-c
 +
 +===== Client =====
 +
 +==== Transfer Files To Client ====
 +
 +****
 +
 +  scp espionage724@192.168.1.153:~/'mediagoblin-user-files-'*'.tar.gz' ~/'mediagoblin-web-files-'*'.tar.gz' ~/'Downloads' && sync
 +
 +====== Restore ======
 +
 +===== Client =====
 +
 +==== Transfer Files To Server ====
 +
 +****
 +
 +  scp ~/'Downloads/mediagoblin-user-files-'*'.tar.gz' ~/'Downloads/mediagoblin-web-files-'*'.tar.gz' espionage724@192.168.1.153:~
 +
 +==== Remove Files ====
 +
 +****
 +
 +  rm -f ~/'Downloads/mediagoblin-user-files-'*'.tar.gz' ~/'Downloads/mediagoblin-web-files-'*'.tar.gz' && sync
 +
 +===== Server =====
 +
 +==== Stop Services ====
 +
 +****
 +
 +  sudo systemctl stop nginx uwsgi media-c
 +
 +==== Remove Previous Folders ====
 +
 +****
 +
 +  sudo rm -Rf '/var/lib/mediagoblin' '/var/www/media'
 +
 +==== Restore GNU MediaGoblin User Folder ====
 +
 +****
 +
 +  cd '/var/lib' && sudo tar -xvzf ~/'mediagoblin-user-files-'*'.tar.gz' 'mediagoblin' && sudo chown -R 'mediagoblin':'www-data' '/var/lib/mediagoblin' && cd ~ && sync
 +
 +==== Restore GNU MediaGoblin Web Folder ====
 +
 +****
 +
 +  cd '/var/www' && sudo tar -xvzf ~/'mediagoblin-web-files-'*'.tar.gz' 'media' && sudo chown -R 'mediagoblin':'www-data' '/var/www/media' && cd ~ && sync
 +
 +==== Start Services ====
 +
 +****
 +
 +  sudo systemctl start nginx uwsgi media-c
 +
 +==== Remove Backups ====
 +
 +  * Verify that GNU MediaGoblin works before running
 +
 +  rm ~/'mediagoblin-user-files-'*'.tar.gz' ~/'mediagoblin-web-files-'*'.tar.gz' && sync
servers/nginx/gnu_mediagoblin.txt ยท Last modified: 2018/09/18 13:31 by Sean Rhone