Changeset - d442d8395b75
[Not reviewed]
default
0 1 0
Thomas De Schampheleire - 6 years ago 2020-08-22 21:22:51
thomas.de_schampheleire@nokia.com
docs: reduce double nesting level in performance.rst

Rendered text was too far to the right.
1 file changed with 12 insertions and 12 deletions:
0 comments (0 inline, 0 general)
docs/usage/performance.rst
Show inline comments
 
@@ -22,104 +22,104 @@ Caching
 

	
 
Tweak beaker cache settings in the ini file. The actual effect of that is
 
questionable.
 

	
 
.. note::
 

	
 
    Beaker has no upper bound on cache size and will never drop any caches. For
 
    memory cache, the only option is to regularly restart the worker process.
 
    For file cache, it must be cleaned manually, as described in the `Beaker
 
    documentation <https://beaker.readthedocs.io/en/latest/sessions.html#removing-expired-old-sessions>`_::
 

	
 
        find data/cache -type f -mtime +30 -print -exec rm {} \;
 

	
 

	
 
Database
 
--------
 

	
 
SQLite is a good option when having a small load on the system. But due to
 
locking issues with SQLite, it is not recommended to use it for larger
 
deployments.
 

	
 
Switching to PostgreSQL or MariaDB/MySQL will result in an immediate performance
 
increase. A tool like SQLAlchemyGrate_ can be used for migrating to another
 
database platform.
 

	
 

	
 
Horizontal scaling
 
------------------
 

	
 
Scaling horizontally means running several Kallithea instances (also known as
 
worker processes) and let them share the load. That is essential to serve other
 
users while processing a long-running request from a user. Usually, the
 
bottleneck on a Kallithea server is not CPU but I/O speed - especially network
 
speed. It is thus a good idea to run multiple worker processes on one server.
 

	
 
.. note::
 

	
 
    Kallithea and the embedded Mercurial backend are not thread-safe. Each
 
    worker process must thus be single-threaded.
 

	
 
Web servers can usually launch multiple worker processes - for example ``mod_wsgi`` with the
 
``WSGIDaemonProcess`` ``processes`` parameter or ``uWSGI`` or ``gunicorn`` with
 
their ``workers`` setting.
 

	
 
Kallithea can also be scaled horizontally across multiple machines.
 
In order to scale horizontally on multiple machines, you need to do the
 
following:
 

	
 
    - Each instance's ``data`` storage needs to be configured to be stored on a
 
      shared disk storage, preferably together with repositories. This ``data``
 
      dir contains template caches, sessions, whoosh index and is used for
 
      task locking (so it is safe across multiple instances). Set the
 
      ``cache_dir``, ``index_dir``, ``beaker.cache.data_dir``, ``beaker.cache.lock_dir``
 
      variables in each .ini file to a shared location across Kallithea instances
 
    - If using several Celery instances,
 
      the message broker should be common to all of them (e.g.,  one
 
      shared RabbitMQ server)
 
    - Load balance using round robin or IP hash, recommended is writing LB rules
 
      that will separate regular user traffic from automated processes like CI
 
      servers or build bots.
 
- Each instance's ``data`` storage needs to be configured to be stored on a
 
  shared disk storage, preferably together with repositories. This ``data``
 
  dir contains template caches, sessions, whoosh index and is used for
 
  task locking (so it is safe across multiple instances). Set the
 
  ``cache_dir``, ``index_dir``, ``beaker.cache.data_dir``, ``beaker.cache.lock_dir``
 
  variables in each .ini file to a shared location across Kallithea instances
 
- If using several Celery instances,
 
  the message broker should be common to all of them (e.g.,  one
 
  shared RabbitMQ server)
 
- Load balance using round robin or IP hash, recommended is writing LB rules
 
  that will separate regular user traffic from automated processes like CI
 
  servers or build bots.
 

	
 

	
 
Serve static files directly from the web server
 
-----------------------------------------------
 

	
 
With the default ``static_files`` ini setting, the Kallithea WSGI application
 
will take care of serving the static files from ``kallithea/public/`` at the
 
root of the application URL.
 

	
 
The actual serving of the static files is very fast and unlikely to be a
 
problem in a Kallithea setup - the responses generated by Kallithea from
 
database and repository content will take significantly more time and
 
resources.
 

	
 
To serve static files from the web server, use something like this Apache config
 
snippet::
 

	
 
        Alias /images/ /srv/kallithea/kallithea/kallithea/public/images/
 
        Alias /css/ /srv/kallithea/kallithea/kallithea/public/css/
 
        Alias /js/ /srv/kallithea/kallithea/kallithea/public/js/
 
        Alias /codemirror/ /srv/kallithea/kallithea/kallithea/public/codemirror/
 
        Alias /fontello/ /srv/kallithea/kallithea/kallithea/public/fontello/
 

	
 
Then disable serving of static files in the ``.ini`` ``app:main`` section::
 

	
 
        static_files = false
 

	
 
If using Kallithea installed as a package, you should be able to find the files
 
under ``site-packages/kallithea``, either in your Python installation or in your
 
virtualenv. When upgrading, make sure to update the web server configuration
 
too if necessary.
 

	
 
It might also be possible to improve performance by configuring the web server
 
to compress responses (served from static files or generated by Kallithea) when
 
serving them. That might also imply buffering of responses - that is more
 
likely to be a problem; large responses (clones or pulls) will have to be fully
 
processed and spooled to disk or memory before the client will see any
 
response. See the documentation for your web server.
 

	
 

	
 
.. _SQLAlchemyGrate: https://github.com/shazow/sqlalchemygrate
 
.. _mod_wsgi: https://modwsgi.readthedocs.io/
 
.. _uWSGI: https://uwsgi-docs.readthedocs.io/
 
.. _gunicorn: http://pypi.python.org/pypi/gunicorn
0 comments (0 inline, 0 general)