2 Commits (35e502d12419df08cfb9deb41d3bdd6e738b1411)

Author	SHA1	Message	Date
Michael Prokop	1e9aa77d53	TT#19059 Port to Python3, update to latest Django LTS version + based on Debian/stretch ... Updates: * Debian jessie->stretch (to use current stable Debian version) * Python 2.7->3.5 (since Python v2 won't be supported with the next Django LTS version any longer and we want to be prepared) * Improve uwsgi configuration for production usage while at it (thanks Florian Apolloner for feedback and suggestions) Necessary changes: * Debian: - we need to use uwsgi-plugin-python3 for Python3 support - replace python-dev with python3-dev for Python3 support and python with python3 accordingly - get rid of __pycache__ directories to avoid inclusion in resulting Debian package * uwsgi: - if we hardcode python34 (for jessie) or python35 (for stretch) then we can't run uwsgi on all supported systems, so instead try `plugin = python3` and rely on update-alternatives mechanism of uwsgi-plugin-python3 Debian package to get it right for, if that shouldn't work as needed then we can look into /usr/share/python3/runtime.d/uwsgi-plugin-python3.rtupdate or shipping repoapi.ini via puppet instead * docker: - python-distribute is not needed (anymore) - adjust package names for Python3 usage, support jessie and stretch at the same time - note: just replacing 'stretch' with 'jessie' in t/Dockerfile is enough to get a working repoapi docker image based on jessie * Python3: - assertItemsEqual became assertCountEqual - mocking: `__builtin__.open` became `builtins.open`, see https://stackoverflow.com/questions/1289894/how-do-i-mock-an-open-used-in-a-with-statement-using-the-mock-framework-in-pyth - we need six >=1.9.0 for raise_from astraction in py3 Otherwise we're failing on Debian/jessie with: | AttributeError: 'module' object has no attribute 'raise_from' See https://github.com/benjaminp/six/blob/master/CHANGES - assert_has_calls changes, fixes: | AssertionError: Calls not found. | Expected: [call('345', 'hotfix fake.git 3.8.7.4+0~mr3.8.7.4 triggered'), | call('123', 'hotfix fake.git 3.8.7.4+0~mr3.8.7.4 triggered')] | Actual: [call('123', 'hotfix fake.git 3.8.7.4+0~mr3.8.7.4 triggered'), | call('345', 'hotfix fake.git 3.8.7.4+0~mr3.8.7.4 triggered')] See https://docs.python.org/dev/library/unittest.mock.html#unittest.mock.Mock.assert_has_calls - urllib: - `from urlparse import urlparse` became `from urllib.parse import urlparse` - `urllib.quote` became `urllib.parse.quote` - configparser: - `from ConfigParser import ...` became `from configparser import ...` * Django: - 'from views import ...' became 'from .views import ...' - the "TEMPLATE_DIRS" setting needs to go into "TEMPLATES = [ { 'DIRS': [ ...." nowadays, see https://docs.djangoproject.com/en/1.11/ref/templates/upgrading/ * celery: - to use latest celery version 4 django-celery needs to be replaced by django-celery-beat + django-celery-results (otherwise we're running into https://github.com/celery/django-celery/issues/497), see documentation at http://docs.celeryproject.org/en/latest/django/first-steps-with-django.html - adjust systemd unit files to use ExecStart | ExecStart=/var/lib/repoapi/venv_prod/bin/celery -A repoapi [...] instead of | ExecStart=/var/lib/repoapi/venv_prod/bin/python3 ./manage.py celery [...] and use `--db=/var/lib/repoapi/flower` instead of `--db=/var/lib/repoapi/flower.db` to avoid ending up with a file named flower.db.db Notes for migration (assuming root permissions): - get rid of existing flower database (not containing relevant data): mv /var/lib/repoapi/flower.db /var/lib/repoapi/flower.db.old - re-deploy virtual environment from scratch: mv /var/lib/repoapi/venv_prod /var/lib/repoapi/venv_prod.old cd /usr/share/repoapi && make deploy - service restarts (in case of virtual env changes): systemctl restart nginx systemctl restart repoapi-beat.service systemctl restart repoapi-flower.service systemctl restart repoapi-worker.service systemctl restart uwsgi - usage instructions for celery + django stuff: /var/lib/repoapi/venv_prod/bin/celery flower --help source /var/lib/repoapi/venv_prod/bin/activate && ./manage.py help --settings="repoapi.settings.prod" - create superuser for django administration (https://repoapi*.mgm.sipwise.com/admin/): source /var/lib/repoapi/venv_prod/bin/activate && ./manage.py createsuperuser --settings="repoapi.settings.prod" Thanks: Victor Seva for assistance with porting Change-Id: Iacb48bf5dadf4eae97b5fbe944e44370e525f64c	9 years ago
Victor Seva	7c335c3b42	TT#18651 provide beat service for periodic tasks ... * purge trunk release Change-Id: I783e45de40d6c6aa69db41c00ed8115a8eacecac	9 years ago