Changeset - 20e850093f1c
[Not reviewed]
default
0 4 0
Mads Kiilerich (mads) - 5 years ago 2021-01-11 23:28:09
mads@kiilerich.com
hooks: on Git, invoke hooks/post-receive-custom from hooks/post-receive

Make it possible for admins to install all kinds of hooks.

Based on a patch by Tim Ooms.

A more generic solution would be nice, but for now we aim for this minimal
solution.
4 files changed with 18 insertions and 3 deletions:
0 comments (0 inline, 0 general)
docs/setup.rst
Show inline comments
 
@@ -303,98 +303,100 @@ This will cause the text ``#300`` to be 
 
The following example transforms a text starting with either of 'pullrequest',
 
'pull request' or 'PR', followed by an optional space, then a pound character
 
(#) and one or more digits, into a link with the text 'PR #' followed by the
 
digits::
 

	
 
    issue_pat = (pullrequest|pull request|PR) ?#(\d+)
 
    issue_server_link = https://issues.example.com/\2
 
    issue_sub = PR #\2
 

	
 
The following example demonstrates how to require whitespace before the issue
 
reference in order for it to be recognized, such that the text ``issue#123`` will
 
not cause a match, but ``issue #123`` will::
 

	
 
    issue_pat = (?:^|(?<=\s))#(\d+)
 
    issue_server_link = https://issues.example.com/\1
 
    issue_sub =
 

	
 
If needed, more than one pattern can be specified by appending a unique suffix to
 
the variables. For example, also demonstrating the use of named groups::
 

	
 
    issue_pat_wiki = wiki-(?P<pagename>\S+)
 
    issue_server_link_wiki = https://wiki.example.com/\g<pagename>
 
    issue_sub_wiki = WIKI-\g<pagename>
 

	
 
With these settings, wiki pages can be referenced as wiki-some-id, and every
 
such reference will be transformed into:
 

	
 
.. code-block:: html
 

	
 
  <a href="https://wiki.example.com/some-id">WIKI-some-id</a>
 

	
 
Refer to the `Python regular expression documentation`_ for more details about
 
the supported syntax in ``issue_pat``, ``issue_server_link`` and ``issue_sub``.
 

	
 

	
 
Hook management
 
---------------
 

	
 
Custom Mercurial hooks can be managed in a similar way to that used in ``.hgrc`` files.
 
To manage hooks, choose *Admin > Settings > Hooks*.
 

	
 
To add another custom hook simply fill in the first textbox with
 
``<name>.<hook_type>`` and the second with the hook path. Example hooks
 
can be found in ``kallithea.lib.hooks``.
 

	
 
Kallithea will also use some hooks internally. They cannot be modified, but
 
some of them can be enabled or disabled in the *VCS* section.
 

	
 
Kallithea has no support for custom Git hooks. Kallithea will install and use
 
Git hooks internally, and they might collide with manually installed hooks.
 
Kallithea does not actively support custom Git hooks, but hooks can be installed
 
manually in the file system. Kallithea will install and use the
 
``post-receive`` Git hook internally, but it will then invoke
 
``post-receive-custom`` if present.
 

	
 

	
 
Changing default encoding
 
-------------------------
 

	
 
By default, Kallithea uses UTF-8 encoding.
 
This is configurable as ``default_encoding`` in the .ini file.
 
This affects many parts in Kallithea including user names, filenames, and
 
encoding of commit messages. In addition Kallithea can detect if the ``chardet``
 
library is installed. If ``chardet`` is detected Kallithea will fallback to it
 
when there are encode/decode errors.
 

	
 
The Mercurial encoding is configurable as ``hgencoding``. It is similar to
 
setting the ``HGENCODING`` environment variable, but will override it.
 

	
 

	
 
Celery configuration
 
--------------------
 

	
 
Kallithea can use the distributed task queue system Celery_ to run tasks like
 
cloning repositories or sending emails.
 

	
 
Kallithea will in most setups work perfectly fine out of the box (without
 
Celery), executing all tasks in the web server process. Some tasks can however
 
take some time to run and it can be better to run such tasks asynchronously in
 
a separate process so the web server can focus on serving web requests.
 

	
 
For installation and configuration of Celery, see the `Celery documentation`_.
 
Note that Celery requires a message broker service like RabbitMQ_ (recommended)
 
or Redis_.
 

	
 
The use of Celery is configured in the Kallithea ini configuration file.
 
To enable it, simply set::
 

	
 
  use_celery = true
 

	
 
and add or change the ``celery.*`` configuration variables.
 

	
 
Configuration settings are prefixed with 'celery.', so for example setting
 
`broker_url` in Celery means setting `celery.broker_url` in the configuration
 
file.
 

	
 
To start the Celery process, run::
 

	
 
  kallithea-cli celery-run -c my.ini
 

	
 
Extra options to the Celery worker can be passed after ``--`` - see ``-- -h``
 
for more info.
docs/upgrade.rst
Show inline comments
 
@@ -191,57 +191,60 @@ Your config file should now work with Al
 

	
 
7. Prepare the front-end
 
------------------------
 

	
 
Starting with Kallithea 0.4, external front-end dependencies are no longer
 
shipped but need to be downloaded and/or generated at installation time. Run the
 
following command::
 

	
 
    kallithea-cli front-end-build
 

	
 

	
 
8. Rebuild the Whoosh full-text index
 
-------------------------------------
 

	
 
It is recommended that you rebuild the Whoosh index after upgrading since
 
new Whoosh versions can introduce incompatible index changes.
 

	
 

	
 
9. Start the Kallithea web application
 
--------------------------------------
 

	
 
This step once again depends entirely on the web server software used to
 
serve Kallithea.
 

	
 
If you were running Kallithea 0.3.x or older and were using ``paster serve
 
my.ini`` before, then the corresponding command in Kallithea 0.4 and later is::
 

	
 
    gearbox serve -c new.ini
 

	
 
Before starting the new version of Kallithea, you may find it helpful to
 
clear out your log file so that new errors are readily apparent.
 

	
 
.. note::
 
    If you're using Celery, make sure you restart all instances of it after
 
    upgrade.
 

	
 

	
 
10. Reinstall internal Git repository hooks
 
-------------------------------------------
 

	
 
It is possible that an upgrade involves changes to the Git hooks installed by
 
Kallithea. As these hooks are created inside the repositories on the server
 
filesystem, they are not updated automatically when upgrading Kallithea itself.
 

	
 
To update the hooks of your Git repositories, run::
 

	
 
    kallithea-cli repo-scan -c my.ini --install-git-hooks
 

	
 
Watch out for warnings like ``skipping overwriting hook file X``, then fix it
 
and rerun, or consider using ``--overwrite-git-hooks`` instead.
 

	
 
Or:
 

	
 
* Go to *Admin > Settings > Remap and Rescan*
 
* Select the checkbox *Install Git hooks*
 
* Click the button *Rescan repositories*
 

	
 
.. note::
 
    Kallithea does not use hooks on Mercurial repositories. This step is thus
 
    not necessary if you only have Mercurial repositories.
docs/usage/troubleshooting.rst
Show inline comments
 
@@ -7,74 +7,75 @@ Troubleshooting
 
:Q: **Missing static files?**
 
:A: Make sure either to set the ``static_files = true`` in the .ini file or
 
   double check the root path for your http setup. It should point to
 
   for example:
 
   ``/home/my-virtual-python/lib/python3.7/site-packages/kallithea/public``
 

	
 
|
 

	
 
:Q: **Can't install celery/rabbitmq?**
 
:A: Don't worry. Kallithea works without them, too. No extra setup is required.
 
    Try out the great Celery docs for further help.
 

	
 
|
 

	
 
:Q: **Long lasting push timeouts?**
 
:A: Make sure you set a longer timeout in your proxy/fcgi settings. Timeouts
 
    are caused by the http server and not Kallithea.
 

	
 
|
 

	
 
:Q: **Large pushes timeouts?**
 
:A: Make sure you set a proper ``max_body_size`` for the http server. Very often
 
    Apache, Nginx, or other http servers kill the connection due to to large
 
    body.
 

	
 
|
 

	
 
:Q: **Apache doesn't pass basicAuth on pull/push?**
 
:A: Make sure you added ``WSGIPassAuthorization true``.
 

	
 
|
 

	
 
:Q: **Git fails on push/pull?**
 
:A: Make sure you're using a WSGI http server that can handle chunked encoding
 
    such as ``waitress`` or ``gunicorn``.
 

	
 
|
 

	
 
:Q: **How can I use hooks in Kallithea?**
 
:A: If using Mercurial, use *Admin > Settings > Hooks* to install
 
    global hooks. Inside the hooks, you can use the current working directory to
 
    control different behaviour for different repositories.
 

	
 
    If using Git, install the hooks manually in each repository, for example by
 
    creating a file ``gitrepo/hooks/pre-receive``.
 
    Note that Kallithea uses the ``post-receive`` hook internally.
 
    Kallithea will not work properly if another post-receive hook is installed instead.
 
    You might also accidentally overwrite your own post-receive hook with the Kallithea hook.
 
    Instead, put your post-receive hook in ``post-receive-custom``, and the Kallithea hook will invoke it.
 

	
 
    You can also use Kallithea-extensions to connect to callback hooks,
 
    for both Git and Mercurial.
 

	
 
|
 

	
 
:Q: **Kallithea is slow for me, how can I make it faster?**
 
:A: See the :ref:`performance` section.
 

	
 
|
 

	
 
:Q: **UnicodeDecodeError on Apache mod_wsgi**
 
:A: Please read: https://docs.djangoproject.com/en/dev/howto/deployment/wsgi/modwsgi/#if-you-get-a-unicodeencodeerror.
 

	
 
|
 

	
 
:Q: **Requests hanging on Windows**
 
:A: Please try out with disabled Antivirus software, there are some known problems with Eset Antivirus. Make sure
 
    you have installed the latest Windows patches (especially KB2789397).
 

	
 

	
 
.. _python: http://www.python.org/
 
.. _mercurial: https://www.mercurial-scm.org/
 
.. _celery: http://celeryproject.org/
 
.. _rabbitmq: http://www.rabbitmq.com/
 
.. _python-ldap: http://www.python-ldap.org/
kallithea/templates/py/git_post_receive_hook.py
Show inline comments
 
"""Kallithea Git hook
 

	
 
This hook is installed and maintained by Kallithea. It will be overwritten
 
by Kallithea - don't customize it manually!
 

	
 
When Kallithea invokes Git, the KALLITHEA_EXTRAS environment variable will
 
contain additional info like the Kallithea instance and user info that this
 
hook will use.
 
"""
 

	
 
import os
 
import subprocess
 
import sys
 

	
 
import kallithea.bin.vcs_hooks
 

	
 

	
 
# Set output mode on windows to binary for stderr.
 
# This prevents python (or the windows console) from replacing \n with \r\n.
 
# Git doesn't display remote output lines that contain \r,
 
# and therefore without this modification git would display empty lines
 
# instead of the exception output.
 
if sys.platform == "win32":
 
    import msvcrt
 
    msvcrt.setmode(sys.stderr.fileno(), os.O_BINARY)
 

	
 
KALLITHEA_HOOK_VER = '_TMPL_'
 
os.environ['KALLITHEA_HOOK_VER'] = KALLITHEA_HOOK_VER
 

	
 

	
 
def main():
 
    repo_path = os.path.abspath('.')
 
    git_stdin_lines = sys.stdin.readlines()
 
    sys.exit(kallithea.bin.vcs_hooks.post_receive(repo_path, git_stdin_lines))
 
    status = kallithea.bin.vcs_hooks.post_receive(repo_path, git_stdin_lines)
 

	
 
    custom_hook = os.path.join(repo_path, 'hooks', 'post-receive-custom')
 
    custom_status = None
 
    if os.access(custom_hook, os.X_OK):
 
        result = subprocess.run([custom_hook], input=''.join(git_stdin_lines), universal_newlines=True)
 
        custom_status = result.returncode
 

	
 
    sys.exit(status or custom_status)
 

	
 

	
 
if __name__ == '__main__':
 
    main()
0 comments (0 inline, 0 general)