* We want to split build releases from hotfix perms
* create dev and devops groups and assign desired permissions.
These groups will match LDAP groups.
* create permissions using hack stolen from
https://stackoverflow.com/questions/29296757/django-data-migrate-permissions
Change-Id: I1b82ebf13aa3cba05e4733e4bef186fbbb52bb13
* don't add '^' twice for api urls
* don't add format_suffix for gerrit/refresh
* add missing format parameter for API views
Change-Id: I2c8cef000c0aec3524302cc5c5a7a201f9cf90d1
When refreshing list of projects, remove every info related to
any old project
We had this scenario when releasing mr10.5, ngcp-cve-scanner was
wrongly added as a project and we couldn't resume the build until
I removed the related info
Change-Id: I58ee97f3256bb63a0a07a6ed606ede0c233cfb84
* trunk-weekly as a normal release, with its own
config file. The idea is to convert eventually
trunk-* too
* match for relase-trunk-* doesn't work anymore
Change-Id: I4967ccc3b5ac47dbebdf6da16d404710fc72da49
* don't allow to trigger more than one build for instance of
release-trunk-* or mrX.Y
* autoupdate pre-commit config
Change-Id: I7183b8645155ca017e9796664d2570e88a29c44e
* calling build from API was always unauthorized
* drfapikey only supports Django until 1.10 version
* https://florimondmanca.github.io/djangorestframework-api-key/
supports modern Python and Django versions
- remove useless BUILD_KEY_AUTH preference, it's True always
Change-Id: I5521b07532dba12abea52982d376eb83293f6a38
* Makefile: add extra test summary
* migrate to structlog, this is an ongoing effort
* panel: remove any "trunk" build
Change-Id: I6f60541248f2970e0fe0784f2eb6eff5dec40721
to be able to build mrX.Y as release-mrX.Y-update
we need to have a previous build. This tools allows
to create a fake build
* don't trigger old BuildRelease instance
* create_build_release() add fake option
Change-Id: I99293a1c41a567d920a086bcaec329defdcf235b
* release has to be 'release-mrX.Y-update' when the build is not the
first
* add check in model, already in view, to don't allow to build more
than one mrX.Y.Z release
* don't trigger copy_debs on release '*-update' builds
* fix BuildRelease.done property, now we can have no
BUILD_RELEASE_JOBS in the list
* BuildReleaseManager.release() in order to get all the BuildRelease
related to the same version now that we have 'release-mrX.Y-update'
Change-Id: I610a246c2f5fc3574153fd226837a060185c379d
* for releases mrX.X and trunk when old builds are done
* add last_update info in the list of builds
* fix error when running tests on local docker
Change-Id: I4912aa00f1a09141ff5d9babfb7b09217097c3d4
* release-copy-debs-yml job expects relase-trunk-$(distribution)
* trigger_builds: use 'trunk' as release parameter
so is clear a trunk build
If we use 'release-trunk-$dist' it gets detected as a release
* repoapi: filter out settings.BUILD_RELEASE_JOBS of the list of projects
when release_uuid is set
Change-Id: I7e522bc05cf4388889fa4aba8828d3254cf983ee
settings.DEBUG doesn't have the proper value defined at settings.dev
use override_settings in BaseTest to fix it and use it on all apps
if necessary
Add a test and mark as expectedFailure to document this
Change-Id: I2017bf695469f4d98db9e5278e43a9e89b5a6be5
This will help us when jumping to Django 2+
> /code/repoapi/urls.py:93: RemovedInDjango20Warning: Specifying a namespace in
> django.conf.urls.include() without providing an app_name is deprecated. Set
> the app_name attribute in the included module, or pass a 2-tuple containing the list of patterns
> and app_name instead.
> url(r"^build/", include("build.urls", namespace="build")),
* py-test: set norecursedirs due to 'build' is in the default
so build app was not included in the test
* py-test: raise errors if warnings from Django are detected
from repoapi
Change-Id: Iab74024a435c1d7851cfe5a51f0ae54a044791a3
* build: don't fail if no builds on done property
* fix progress stats on panel_uuid
* stop sending queries if release is done
Change-Id: I4fb4c6c161a852bb8f5ee978cec053d98e38a1f8
* added mr8.1.2 and data to test it
* don't add project as failed for piuparts job with FAILURE status
Change-Id: I95a8bdbf5a73b4f353d99bc172f879a90870e1f0
* list of supported releases is only going to grow
better to group them
* add some more config yaml files to able to produce
groups in test
Change-Id: I13fa7e6f3de08b5f131dc5c19cfd28d0ed7a4f53
* smart build release reading per release metadata file
provided by sipwise-repos-scripts-data package
* build release will trigger settings.RELEASE_JOBS jobs first
* old build process hidden at /release_panel/old/
as a fallback just in case
* some examples of config/<release>.yml files
just for test
* style changes by black pre-commit plugin to unify
code style
* rework panel.js to support both views /release/ and /release_uuid/
via $.release.uuid value
* cosmetic improvements
Change-Id: I5f581100e6c297a1cc0abc4bb1ddf978870b6936
Attempt to fix problem when release panel triggers jenkins jobs
setting branch=none when tag=mrX.X.X.1, when the branch parameter
should have branch=mrX.X.X
A change from string.join() to "".join() is needed for Python 3.
Change-Id: I89f7bb304180880bc7317b61aa675b8d2162f091
Attempt to fix problem when release panel triggers jenkins jobs
setting branch=none when tag=mrX.X.X.1, when the branch parameter
should have branch=mrX.X.X
Change-Id: Ic510cec2e94400a5c5a3c8bb0935ff19cdf0ce17
Updates:
* Debian jessie->stretch (to use current stable Debian version)
* Python 2.7->3.5 (since Python v2 won't be supported with the
next Django LTS version any longer and we want to be prepared)
* Improve uwsgi configuration for production usage while at it
(thanks Florian Apolloner for feedback and suggestions)
Necessary changes:
* Debian:
- we need to use uwsgi-plugin-python3 for Python3 support
- replace python-dev with python3-dev for Python3 support
and python with python3 accordingly
- get rid of __pycache__ directories to avoid inclusion in
resulting Debian package
* uwsgi:
- if we hardcode python34 (for jessie) or python35 (for stretch)
then we can't run uwsgi on all supported systems, so instead
try `plugin = python3` and rely on update-alternatives mechanism
of uwsgi-plugin-python3 Debian package to get it right for,
if that shouldn't work as needed then we can look into
/usr/share/python3/runtime.d/uwsgi-plugin-python3.rtupdate
or shipping repoapi.ini via puppet instead
* docker:
- python-distribute is not needed (anymore)
- adjust package names for Python3 usage, support jessie
and stretch at the same time
- note: just replacing 'stretch' with 'jessie' in t/Dockerfile
is enough to get a working repoapi docker image based on jessie
* Python3:
- assertItemsEqual became assertCountEqual
- mocking: `__builtin__.open` became `builtins.open`, see
https://stackoverflow.com/questions/1289894/how-do-i-mock-an-open-used-in-a-with-statement-using-the-mock-framework-in-pyth
- we need six >=1.9.0 for raise_from astraction in py3
Otherwise we're failing on Debian/jessie with:
| AttributeError: 'module' object has no attribute 'raise_from'
See https://github.com/benjaminp/six/blob/master/CHANGES
- assert_has_calls changes, fixes:
| AssertionError: Calls not found.
| Expected: [call('345', 'hotfix fake.git 3.8.7.4+0~mr3.8.7.4 triggered'),
| call('123', 'hotfix fake.git 3.8.7.4+0~mr3.8.7.4 triggered')]
| Actual: [call('123', 'hotfix fake.git 3.8.7.4+0~mr3.8.7.4 triggered'),
| call('345', 'hotfix fake.git 3.8.7.4+0~mr3.8.7.4 triggered')]
See https://docs.python.org/dev/library/unittest.mock.html#unittest.mock.Mock.assert_has_calls
- urllib:
- `from urlparse import urlparse` became `from urllib.parse import urlparse`
- `urllib.quote` became `urllib.parse.quote`
- configparser:
- `from ConfigParser import ...` became `from configparser import ...`
* Django:
- 'from views import ...' became 'from .views import ...'
- the "TEMPLATE_DIRS" setting needs to go into "TEMPLATES = [ { 'DIRS': [ ...."
nowadays, see https://docs.djangoproject.com/en/1.11/ref/templates/upgrading/
* celery:
- to use latest celery version 4 django-celery needs to be replaced
by django-celery-beat + django-celery-results
(otherwise we're running into https://github.com/celery/django-celery/issues/497),
see documentation at http://docs.celeryproject.org/en/latest/django/first-steps-with-django.html
- adjust systemd unit files to use ExecStart
| ExecStart=/var/lib/repoapi/venv_prod/bin/celery -A repoapi [...]
instead of
| ExecStart=/var/lib/repoapi/venv_prod/bin/python3 ./manage.py celery [...]
and use `--db=/var/lib/repoapi/flower` instead of
`--db=/var/lib/repoapi/flower.db` to avoid ending up with a file
named flower.db.db
Notes for migration (assuming root permissions):
- get rid of existing flower database (not containing relevant data):
mv /var/lib/repoapi/flower.db /var/lib/repoapi/flower.db.old
- re-deploy virtual environment from scratch:
mv /var/lib/repoapi/venv_prod /var/lib/repoapi/venv_prod.old
cd /usr/share/repoapi && make deploy
- service restarts (in case of virtual env changes):
systemctl restart nginx
systemctl restart repoapi-beat.service
systemctl restart repoapi-flower.service
systemctl restart repoapi-worker.service
systemctl restart uwsgi
- usage instructions for celery + django stuff:
/var/lib/repoapi/venv_prod/bin/celery flower --help
source /var/lib/repoapi/venv_prod/bin/activate && ./manage.py help --settings="repoapi.settings.prod"
- create superuser for django administration (https://repoapi*.mgm.sipwise.com/admin/):
source /var/lib/repoapi/venv_prod/bin/activate && ./manage.py createsuperuser --settings="repoapi.settings.prod"
Thanks: Victor Seva for assistance with porting
Change-Id: Iacb48bf5dadf4eae97b5fbe944e44370e525f64c
Victor Seva
Manuel Montecelo
Michael Prokop