kallithea Changeset - 01cb988e82a5

Changeset - 01cb988e82a5

Parent rev.

Child rev.

[Not reviewed]

default

0 2 0

Mads Kiilerich (mads) - 5 years ago 2021-01-01 18:04:16
mads@kiilerich.com

celery: celery-run should only initialize app and sqlalchemy after workers have been forked

If app and SqlAlchemy were initialized before launching celery, the forked
workers would inherit the database connection ... and that doesn't work.

This could be handled by disposing the engine after forking the worker or
before each task ... but it remains unnecessary and wrong to initialize the
engine early when it isn't used, and then fork it.

2 files changed with 5 insertions and 3 deletions:

kallithea/bin/kallithea_cli_celery.py

kallithea/config/app_cfg.py

0 comments (0 inline, 0 general)

kallithea/bin/kallithea_cli_celery.py

➞

Show inline comments

@@ @@ -14,12 +14,13 @@ @@
 import celery.bin.worker
 import click
 import kallithea
 import kallithea.bin.kallithea_cli_base as cli_base
 from kallithea.lib import celery_app
 from kallithea.lib.utils2 import asbool
 @cli_base.register_command(needs_config_file=True)
 @click.argument('celery_args', nargs=-1)
 def celery_run(celery_args, config):
@@ @@ -34,11 +35,12 @@ def celery_run(celery_args, config): @@
     """
     if not asbool(config.get('use_celery')):
         raise Exception('Please set use_celery = true in .ini config '
                         'file before running this command')
     # do as config_file_initialize_app
     kallithea.config.application.make_app(config.global_conf, **config.local_conf)
     kallithea.CELERY_APP.config_from_object(celery_app.make_celery_config(config))
     kallithea.CELERY_APP.loader.on_worker_process_init = lambda: kallithea.config.application.make_app(config.global_conf, **config.local_conf)
     cmd = celery.bin.worker.worker(kallithea.CELERY_APP)
     return cmd.run_from_argv(None, command='celery-run -c CONFIG_FILE --', argv=list(celery_args))

kallithea/config/app_cfg.py

➞

Show inline comments

@@ @@ -132,13 +132,13 @@ def setup_configuration(app): @@
                       'to your .ini file to skip the check.\n' % (' '.join(current_heads), ' '.join(available_heads)))
             sys.exit(1)
     # store some globals into kallithea
     kallithea.DEFAULT_USER_ID = db.User.get_default_user().user_id
     if asbool(config.get('use_celery')):
+    if asbool(config.get('use_celery')) and not kallithea.CELERY_APP.finalized:
         kallithea.CELERY_APP.config_from_object(celery_app.make_celery_config(config))
     kallithea.CONFIG = config
     load_extensions(root_path=config['here'])
     set_app_settings(config)

0 comments (0 inline, 0 general)