When invoking `virtualenv --python=python3 virtualenv` to deploy the
virtual environment, installing the python-debian wheel, and then
re-executing `virtualenv --python=python3 virtualenv` once again, we
might end up without the virtualenv/lib/python*/site-packages/debian
directory, resulting in:
| Traceback (most recent call last):
| File "<string>", line 1, in <module>
| ModuleNotFoundError: No module named 'debian'
This is a bug in setuptools >=51.3.3-1, see
https://bugs.debian.org/1041091 for the details, so currently affecting
all systems with Debian/bookworm.
By excluding installation of setuptools (via --no-setuptools), we would
be able to work around this bootstrapping problem itself, though then
installation of django-jsonify (which we depend on) fails with:
| ModuleNotFoundError: No module named 'setuptools'
Let's take this as a chance to switch from virtualenv to the more
lightweight venv, quoting https://docs.python.org/3/library/venv.html:
| Changed in version 3.5: The use of venv is now recommended for creating virtual environments.
Change-Id: Iad06bf8c49abd96441ab04bb9b7e3917acd1ee59
* set default tracker to mantis, None for test/dev
* debian: add missing sipwise-workfront-tools dependency
Added as Recomends since we are not going to use it
in the near future
* MANTIS_* settings
Change-Id: Id7b19f23d316a31b362112167edb5c395baf554b
Set debhelper compat level in Build-Depends instead of debian/compat.
Do not explicitly request the systemd sequence.
Use dh_installsystemd instead of deprecated dh_systemd_*.
Change-Id: Ic8cbf795de90aa8900a899706ce6e0474a8e6eed
- Bump debhelper compatibility version 10.
- Switch from Priority extra to optional.
- New debian/.gitignore file.
- Canonicalize Homepage URL.
- Remove dh-make boilerplate in debian/rules.
Change-Id: Ic731001d847b143ae9d0bbc6ca23202b39003f67
Updates:
* Debian jessie->stretch (to use current stable Debian version)
* Python 2.7->3.5 (since Python v2 won't be supported with the
next Django LTS version any longer and we want to be prepared)
* Improve uwsgi configuration for production usage while at it
(thanks Florian Apolloner for feedback and suggestions)
Necessary changes:
* Debian:
- we need to use uwsgi-plugin-python3 for Python3 support
- replace python-dev with python3-dev for Python3 support
and python with python3 accordingly
- get rid of __pycache__ directories to avoid inclusion in
resulting Debian package
* uwsgi:
- if we hardcode python34 (for jessie) or python35 (for stretch)
then we can't run uwsgi on all supported systems, so instead
try `plugin = python3` and rely on update-alternatives mechanism
of uwsgi-plugin-python3 Debian package to get it right for,
if that shouldn't work as needed then we can look into
/usr/share/python3/runtime.d/uwsgi-plugin-python3.rtupdate
or shipping repoapi.ini via puppet instead
* docker:
- python-distribute is not needed (anymore)
- adjust package names for Python3 usage, support jessie
and stretch at the same time
- note: just replacing 'stretch' with 'jessie' in t/Dockerfile
is enough to get a working repoapi docker image based on jessie
* Python3:
- assertItemsEqual became assertCountEqual
- mocking: `__builtin__.open` became `builtins.open`, see
https://stackoverflow.com/questions/1289894/how-do-i-mock-an-open-used-in-a-with-statement-using-the-mock-framework-in-pyth
- we need six >=1.9.0 for raise_from astraction in py3
Otherwise we're failing on Debian/jessie with:
| AttributeError: 'module' object has no attribute 'raise_from'
See https://github.com/benjaminp/six/blob/master/CHANGES
- assert_has_calls changes, fixes:
| AssertionError: Calls not found.
| Expected: [call('345', 'hotfix fake.git 3.8.7.4+0~mr3.8.7.4 triggered'),
| call('123', 'hotfix fake.git 3.8.7.4+0~mr3.8.7.4 triggered')]
| Actual: [call('123', 'hotfix fake.git 3.8.7.4+0~mr3.8.7.4 triggered'),
| call('345', 'hotfix fake.git 3.8.7.4+0~mr3.8.7.4 triggered')]
See https://docs.python.org/dev/library/unittest.mock.html#unittest.mock.Mock.assert_has_calls
- urllib:
- `from urlparse import urlparse` became `from urllib.parse import urlparse`
- `urllib.quote` became `urllib.parse.quote`
- configparser:
- `from ConfigParser import ...` became `from configparser import ...`
* Django:
- 'from views import ...' became 'from .views import ...'
- the "TEMPLATE_DIRS" setting needs to go into "TEMPLATES = [ { 'DIRS': [ ...."
nowadays, see https://docs.djangoproject.com/en/1.11/ref/templates/upgrading/
* celery:
- to use latest celery version 4 django-celery needs to be replaced
by django-celery-beat + django-celery-results
(otherwise we're running into https://github.com/celery/django-celery/issues/497),
see documentation at http://docs.celeryproject.org/en/latest/django/first-steps-with-django.html
- adjust systemd unit files to use ExecStart
| ExecStart=/var/lib/repoapi/venv_prod/bin/celery -A repoapi [...]
instead of
| ExecStart=/var/lib/repoapi/venv_prod/bin/python3 ./manage.py celery [...]
and use `--db=/var/lib/repoapi/flower` instead of
`--db=/var/lib/repoapi/flower.db` to avoid ending up with a file
named flower.db.db
Notes for migration (assuming root permissions):
- get rid of existing flower database (not containing relevant data):
mv /var/lib/repoapi/flower.db /var/lib/repoapi/flower.db.old
- re-deploy virtual environment from scratch:
mv /var/lib/repoapi/venv_prod /var/lib/repoapi/venv_prod.old
cd /usr/share/repoapi && make deploy
- service restarts (in case of virtual env changes):
systemctl restart nginx
systemctl restart repoapi-beat.service
systemctl restart repoapi-flower.service
systemctl restart repoapi-worker.service
systemctl restart uwsgi
- usage instructions for celery + django stuff:
/var/lib/repoapi/venv_prod/bin/celery flower --help
source /var/lib/repoapi/venv_prod/bin/activate && ./manage.py help --settings="repoapi.settings.prod"
- create superuser for django administration (https://repoapi*.mgm.sipwise.com/admin/):
source /var/lib/repoapi/venv_prod/bin/activate && ./manage.py createsuperuser --settings="repoapi.settings.prod"
Thanks: Victor Seva for assistance with porting
Change-Id: Iacb48bf5dadf4eae97b5fbe944e44370e525f64c
* update dependencies due postgresql
* fake files needed to deploy at debian
* add new needed config values at server.ini file
Change-Id: I02699c721ac79a0328da07a4c2e3a2cb1451d403
* remove uwsgi configs for them
* manage secret_key and virtualenv on postinst
* restart uwsgi on postinst
* add missing uwsgi as dependency
Change-Id: I40a85e75d3961c7e84d49fce5e3026064cac1d56
* python-debian installed via pip
We are using virtualenv on production, so no need to
install packages for that
Change-Id: I2db86ffa9e71814b1d6dc9d7675bc9790b2c7ee7
* release-tools-runner sends a note
"hotfix <projectname> <version> triggered"
* TODO: how to be sure that version reaches
public release ??
Change-Id: Ib220f36d642172d8c89f4df35543193be16d37ce
- install repoapi.ini file into uwsgi/apps-available
- venv_prod should be at /var/lib/repoapi
- wrap-and-short -a
Change-Id: I31c491103c35b22ee585424d739e7afe8db7249d
Guillem Jover
Michael Prokop
Victor Seva