def createvirtualenv = ''

def activatevirtualenv = ''

node {

    properties([[$class: 'BuildDiscarderProperty',

                  strategy: [$class: 'LogRotator',

                              artifactDaysToKeepStr: '',

                              artifactNumToKeepStr: '10',

                              daysToKeepStr: '',

                              numToKeepStr: '']]]);

    if (isUnix()) {

        activatevirtualenv = '. $JENKINS_HOME/venv/$JOB_NAME/bin/activate'

    } else {

        activatevirtualenv = 'call %JENKINS_HOME%\\venv\\%JOB_NAME%\\Scripts\\activate.bat'

    }

    stage('checkout') {

        checkout scm

        if (isUnix()) {

            sh 'hg --config extensions.purge= purge --all'

        } else {

            bat 'hg --config extensions.purge= purge --all'

        }

    }

    stage('virtual env') {

        def virtualenvscript = """$createvirtualenv

            $activatevirtualenv

            python -m pip install --upgrade pip

            pip install --upgrade setuptools

            pip install --upgrade pylint

            pip install --upgrade pytest-cov

            """

        if (isUnix()) {

            virtualenvscript += """

                pip install --upgrade python-ldap

                pip install --upgrade python-pam

                """

About

-----

**Kallithea** is a fast and powerful management tool for Mercurial_ and Git_

with a built-in push/pull server, full text search and code-review. It works on

HTTP/HTTPS and SSH, has a built-in permission/authentication system with the ability

to authenticate via LDAP or ActiveDirectory. Kallithea also provides simple API

so it's easy to integrate with existing external systems.

Kallithea is similar in some respects to GitHub_ or Bitbucket_, however

Kallithea can be run as standalone hosted application on your own server. It is

open-source and focuses more on providing a customised,

self-administered interface for Mercurial_ and Git_ repositories. Kallithea

works on Unix-like systems and Windows.

Kallithea was forked from RhodeCode in July 2014 and has been heavily modified.

Installation

------------

Kallithea requires Python_ 2.7 and it is recommended to install it in a

    pip install kallithea

The development repository is kept very stable and used in production by the

developers -- you can do the same.

Please visit https://docs.kallithea-scm.org/en/latest/installation.html for

more details.

There is also an experimental `Puppet module`_ for installing and setting up

Kallithea. Currently, only basic functionality is provided, but it is still

enough to get up and running quickly, especially for people without Python

background. See

https://docs.kallithea-scm.org/en/latest/installation_puppet.html for further

information.

Source code

-----------

The latest sources can be obtained from

https://kallithea-scm.org/repos/kallithea.

The issue tracker and a repository mirror can be found at Bitbucket_ on

Online documentation for the current version of Kallithea is available at

https://docs.kallithea-scm.org/en/stable/. Documentation for the current development

version can be found on https://docs.kallithea-scm.org/en/default/.

You can also build the documentation locally: go to ``docs/`` and run::

   make html

.. note:: You need to have Sphinx_ installed to build the

          documentation. If you don't have Sphinx_ installed you can

          install it via the command: ``pip install sphinx`` .

Migrating from RhodeCode

------------------------

Kallithea 0.3.2 and earlier supports migrating from an existing RhodeCode

installation. To migrate, install Kallithea 0.3.2 and follow the

instructions in the 0.3.2 README to perform a one-time conversion of the

database from RhodeCode to Kallithea, before upgrading to this version

of Kallithea.

.. _Python: http://www.python.org/

.. _Sphinx: http://sphinx.pocoo.org/

.. _Mercurial: http://mercurial.selenic.com/

.. _Bitbucket: http://bitbucket.org/

.. _GitHub: http://github.com/

.. _Subversion: http://subversion.tigris.org/

.. _Git: http://git-scm.com/

.. _Celery: http://celeryproject.org/

.. _Software Freedom Conservancy: http://sfconservancy.org/

.. _Puppet module: https://forge.puppetlabs.com/rauch/kallithea

Infrastructure

--------------

The main repository is hosted on Our Own Kallithea (aka OOK) at

https://kallithea-scm.org/repos/kallithea/, our self-hosted instance

of Kallithea.

For now, we use Bitbucket_ for `pull requests`_ and `issue tracking`_. The

issue tracker is for tracking bugs, not for support, discussion, or ideas --

please use the `mailing list`_ or :ref:`IRC <readme>` to reach the community.

We use Weblate_ to translate the user interface messages into languages other

than English. Join our project on `Hosted Weblate`_ to help us.

To register, you can use your Bitbucket or GitHub account. See :ref:`translations`

for more details.

Getting started

---------------

To get started with Kallithea development::

        hg clone https://kallithea-scm.org/repos/kallithea

        cd kallithea

        source ../kallithea-venv/bin/activate

        pip install --upgrade pip setuptools

        pip install --upgrade -e . -r dev_requirements.txt python-ldap python-pam

        kallithea-cli config-create my.ini

        kallithea-cli db-create -c my.ini --user=user --email=user@example.com --password=password --repos=/tmp

        kallithea-cli front-end-build

        gearbox serve -c my.ini --reload &

        firefox http://127.0.0.1:5000/

If you plan to use Bitbucket_ for sending contributions, you can also fork

Kallithea on Bitbucket_ first (https://bitbucket.org/conservancy/kallithea) and

then replace the clone step above by a clone of your fork. In this case, please

see :ref:`contributing-guidelines` below for configuring your fork correctly.

Contribution flow

-----------------

Starting from an existing Kallithea clone, make sure it is up to date with the

latest upstream changes::

        hg pull

        hg update

User guide

**********

.. toctree::

   :maxdepth: 1

   usage/general

   usage/vcs_notes

   usage/statistics

   api/api

Developer guide

***************

.. toctree::

   :maxdepth: 1

   contributing

   dev/translation

   dev/dbmigrations

.. _python: http://www.python.org/

.. _django: http://www.djangoproject.com/

.. _mercurial: https://www.mercurial-scm.org/

.. _bitbucket: http://bitbucket.org/

.. _subversion: http://subversion.tigris.org/

.. _git: http://git-scm.com/

.. _celery: http://celeryproject.org/

.. _Sphinx: http://sphinx.pocoo.org/

.. _vcs: http://pypi.python.org/pypi/vcs

  needed by other packages.

Regardless of the installation method you may need to make sure you have

appropriate development packages installed, as installation of some of the

Kallithea dependencies requires a working C compiler and libffi library

headers. Depending on your configuration, you may also need to install

Git and development packages for the database of your choice.

For Debian and Ubuntu, the following command will ensure that a reasonable

set of dependencies is installed::

    sudo apt-get install build-essential git python-pip python-virtualenv libffi-dev python-dev

For Fedora and RHEL-derivatives, the following command will ensure that a

reasonable set of dependencies is installed::

    sudo yum install gcc git python-pip python-virtualenv libffi-devel python-devel

.. _installation-source:

Installation from repository source

-----------------------------------

repository, follow the instructions below::

        hg clone https://kallithea-scm.org/repos/kallithea -u stable

        cd kallithea

        . ../kallithea-venv/bin/activate

        pip install --upgrade pip setuptools

        pip install --upgrade -e .

        python2 setup.py compile_catalog   # for translation of the UI

You can now proceed to :ref:`setup`.

.. _installation-virtualenv:

Installing a released version in a virtualenv

---------------------------------------------

This way, all libraries required by Kallithea will be installed separately from your

main Python installation and other applications and things will be less

problematic when upgrading the system or Kallithea.

  basic requirements are up-to-date by running::

    . /srv/kallithea/venv/bin/activate

    pip install --upgrade pip setuptools

.. note:: You can't use UNIX ``sudo`` to source the ``virtualenv`` script; it

   will "activate" a shell that terminates immediately. It is also perfectly

   acceptable (and desirable) to create a virtualenv as a normal user.

- Make a folder for Kallithea data files, and configuration somewhere on the

  filesystem. For example::

    mkdir /srv/kallithea

- Go into the created directory and run this command to install Kallithea::

    pip install --upgrade kallithea

.. note:: Some dependencies are optional. If you need them, install them in

   the virtualenv too::

     pip install --upgrade kallithea python-ldap python-pam psycopg2

   This might require installation of development packages using your

- This will install Kallithea together with all other required

  Python libraries into the activated virtualenv.

You can now proceed to :ref:`setup`.

.. _installation-without-virtualenv:

Installing a released version without virtualenv

------------------------------------------------

For installation without virtualenv, 'just' use::

    pip install kallithea

Note that this method requires root privileges and will install packages

globally without using the system's package manager.

To install as a regular user in ``~/.local``, you can use::

    pip install --user kallithea

You can now proceed to :ref:`setup`.

Step 5 -- Kallithea folder structure

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Create a Kallithea folder structure.

This is only an example to install Kallithea. Of course, you can

change it. However, this guide will follow the proposed structure, so

please later adapt the paths if you change them. Folders without

spaces are recommended.

Create the following folder structure::

  C:\Kallithea

  C:\Kallithea\Bin

  C:\Kallithea\Env

  C:\Kallithea\Repos

Step 6 -- Install virtualenv

^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. note::

   A python virtual environment will allow for isolation between the Python packages of your system and those used for Kallithea.

   It is strongly recommended to use it to ensure that Kallithea does not change a dependency that other software uses or vice versa.

To create a virtual environment, run::

Step 7 -- Install Kallithea

^^^^^^^^^^^^^^^^^^^^^^^^^^^

In order to install Kallithea, you need to be able to run "pip install kallithea". It will use pip to install the Kallithea Python package and its dependencies.

Some Python packages use managed code and need to be compiled.

This can be done on Linux without any special steps. On Windows, you will need to install Microsoft Visual C++ compiler for Python 2.7.

Download and install "Microsoft Visual C++ Compiler for Python 2.7" from http://aka.ms/vcpython27

.. note::

  You can also install the dependencies using already compiled Windows binaries packages. A good source of compiled Python packages is http://www.lfd.uci.edu/~gohlke/pythonlibs/. However, not all of the necessary packages for Kallithea are on this site and some are hard to find, so we will stick with using the compiler.

In a command prompt type (adapting paths if necessary)::

  cd C:\Kallithea\Env\Scripts

  activate

  pip install --upgrade pip setuptools

The prompt will change into "(Env) C:\\Kallithea\\Env\\Scripts" or similar

(depending of your folder structure). Then type::

  pip install kallithea

  Please substitute [your-python-path] with your Python installation path.

  Typically: C:\\Python27

Step 5 -- Kallithea folder structure

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Create a Kallithea folder structure

This is only a example to install Kallithea, you can of course change

it. However, this guide will follow the proposed structure, so please

later adapt the paths if you change them. My recommendation is to use

folders with NO SPACES. But you can try if you are brave...

Create the following folder structure::

  C:\Kallithea

  C:\Kallithea\Bin

  C:\Kallithea\Env

  C:\Kallithea\Repos

Step 6 -- Install virtualenv

^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Step 7 -- Install Kallithea

^^^^^^^^^^^^^^^^^^^^^^^^^^^

Finally, install Kallithea

Close previously opened command prompt/s, and open a Visual Studio 2008

Command Prompt (**IMPORTANT!!**). To do so, go to Start Menu, and then open

"Microsoft Visual C++ 2008 Express Edition" -> "Visual Studio Tools" ->

"Visual Studio 2008 Command Prompt"

.. note::

   64-bit: For 64-bit you need to modify the shortcut that is used to start the

   Visual Studio 2008 Command Prompt. Use right-mouse click to open properties.

Change commandline from::

%comspec% /k ""C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\vcvarsall.bat"" x86

to::

%comspec% /k ""C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\vcvarsall.bat"" amd64

Python environment

------------------

**Kallithea** is written entirely in Python_ and requires Python version

2.7 or higher. Python 3.x is currently not supported.

Given a Python installation, there are different ways of providing the

environment for running Python applications. Each of them pretty much

corresponds to a ``site-packages`` directory somewhere where packages can be

installed.

Kallithea itself can be run from source or be installed, but even when running

from source, there are some dependencies that must be installed in the Python

environment used for running Kallithea.

- Packages *could* be installed in Python's ``site-packages`` directory ... but

  that would require running pip_ as root and it would be hard to uninstall or

  upgrade and is probably not a good idea unless using a package manager.

- Packages could also be installed in ``~/.local`` ... but that is probably

  only a good idea if using a dedicated user per application or instance.

  "container" where each Kallithea instance can get its own dedicated and

  self-contained virtual environment.

We recommend using virtualenv for installing Kallithea.

Locale environment

------------------

In order to ensure a correct functioning of Kallithea with respect to non-ASCII

characters in user names, file paths, commit messages, etc., it is very

important that Kallithea is run with a correct `locale` configuration.

On Unix, environment variables like ``LANG`` or ``LC_ALL`` can specify a language (like

``en_US``) and encoding (like ``UTF-8``) to use for code points outside the ASCII

range. The flexibility of supporting multiple encodings of Unicode has the flip

side of having to specify which encoding to use - especially for Mercurial.

It depends on the OS distribution and system configuration which locales are

available. For example, some Docker containers based on Debian default to only

supporting the ``C`` language, while other Linux environments have ``en_US`` but not

``C``. The ``locale -a`` command will show which values are available on the

current system. Regardless of the actual language, you should normally choose a

locale that has the ``UTF-8`` encoding (note that spellings ``utf8``, ``utf-8``,

- uWSGI_ is also a full web server with built-in WSGI module.

- IIS_ can also server WSGI applications directly using isapi-wsgi_.

- A `reverse HTTP proxy <https://en.wikipedia.org/wiki/Reverse_proxy>`_

  can be put in front of another web server which has WSGI support.

  Such a layered setup can be complex but might in some cases be the right

  option, for example to standardize on one internet-facing web server, to add

  encryption or special authentication or for other security reasons, to

  provide caching of static files, or to provide load balancing or fail-over.

  Nginx_, Varnish_ and HAProxy_ are often used for this purpose, often in front

  of a ``gearbox serve`` that somehow is wrapped as a service.

The best option depends on what you are familiar with and the requirements for

performance and stability. Also, keep in mind that Kallithea mainly is serving

dynamically generated pages from a relatively slow Python process. Kallithea is

also often used inside organizations with a limited amount of users and thus no

continuous hammering from the internet.

.. _Python: http://www.python.org/

.. _Gunicorn: http://gunicorn.org/

.. _Waitress: http://waitress.readthedocs.org/en/latest/

.. _Gearbox: http://turbogears.readthedocs.io/en/latest/turbogears/gearbox.html

.. _PyPI: https://pypi.python.org/pypi

.. _Apache httpd: http://httpd.apache.org/

.. _mod_wsgi: https://code.google.com/p/modwsgi/

.. _isapi-wsgi: https://github.com/hexdump42/isapi-wsgi

.. _uWSGI: https://uwsgi-docs.readthedocs.org/en/latest/

.. _nginx: http://nginx.org/en/

.. _iis: http://en.wikipedia.org/wiki/Internet_Information_Services

.. _pip: http://en.wikipedia.org/wiki/Pip_%28package_manager%29

.. _WSGI: http://en.wikipedia.org/wiki/Web_Server_Gateway_Interface

.. _HAProxy: http://www.haproxy.org/

.. _Varnish: https://www.varnish-cache.org/

.. _npm: https://www.npmjs.com/

          python-home=/srv/kallithea/venv lang=C.UTF-8

      WSGIProcessGroup kallithea

      WSGIScriptAlias / /srv/kallithea/dispatch.wsgi

      WSGIPassAuthorization On

  Or if using a dispatcher WSGI script with proper virtualenv activation:

  .. code-block:: apache

      WSGIDaemonProcess kallithea processes=5 threads=1 maximum-requests=100 lang=en_US.utf8

      WSGIProcessGroup kallithea

      WSGIScriptAlias / /srv/kallithea/dispatch.wsgi

      WSGIPassAuthorization On

Other configuration files

-------------------------

A number of `example init.d scripts`__ can be found in

the ``init.d`` directory of the Kallithea source.

.. __: https://kallithea-scm.org/repos/kallithea/files/tip/init.d/ .

.. _python: http://www.python.org/

.. _Python regular expression documentation: https://docs.python.org/2/library/re.html

.. _Mercurial: https://www.mercurial-scm.org/

.. _Celery: http://celeryproject.org/

.. _Celery documentation: http://docs.celeryproject.org/en/latest/getting-started/index.html

.. _RabbitMQ: http://www.rabbitmq.com/

.. _Redis: http://redis.io/

.. _mercurial-server: http://www.lshift.net/mercurial-server.html

.. _PublishingRepositories: https://www.mercurial-scm.org/wiki/PublishingRepositories

.. _WSGIDaemonProcess documentation: https://modwsgi.readthedocs.io/en/develop/configuration-directives/WSGIDaemonProcess.html

Before starting the new version of Kallithea, you may find it helpful to

clear out your log file so that new errors are readily apparent.

.. note::

    If you're using Celery, make sure you restart all instances of it after

    upgrade.

10. Update Git repository hooks

-------------------------------

It is possible that an upgrade involves changes to the Git hooks installed by

Kallithea. As these hooks are created inside the repositories on the server

filesystem, they are not updated automatically when upgrading Kallithea itself.

To update the hooks of your Git repositories:

* Go to *Admin > Settings > Remap and Rescan*

* Select the checkbox *Install Git hooks*

* Click the button *Rescan repositories*

.. note::

    Kallithea does not use hooks on Mercurial repositories. This step is thus

    not necessary if you only have Mercurial repositories.

:A: It's easy if they are Python hooks: just use advanced link in

    hooks section in Admin panel, that works only for Mercurial. If

    you want to use Git hooks, just install th proper one in the repository,

    e.g., create a file `/gitrepo/hooks/pre-receive`. You can also use

    Kallithea-extensions to connect to callback hooks, for both Git

    and Mercurial.

|

:Q: **Kallithea is slow for me, how can I make it faster?**

:A: See the :ref:`performance` section.

|

:Q: **UnicodeDecodeError on Apache mod_wsgi**

:A: Please read: https://docs.djangoproject.com/en/dev/howto/deployment/wsgi/modwsgi/#if-you-get-a-unicodeencodeerror.

|

:Q: **Requests hanging on Windows**

:A: Please try out with disabled Antivirus software, there are some known problems with Eset Antivirus. Make sure

    you have installed the latest Windows patches (especially KB2789397).

.. _python: http://www.python.org/

.. _mercurial: https://www.mercurial-scm.org/

.. _celery: http://celeryproject.org/

.. _rabbitmq: http://www.rabbitmq.com/

.. _python-ldap: http://www.python-ldap.org/

#!/bin/bash

set -e

set -x

cleanup()

{

  echo "Removing venv $venv"

  rm  -rf "$venv"

}

echo "Checking that you are NOT inside a virtualenv"

[ -z "$VIRTUAL_ENV" ]

venv=$(mktemp -d --tmpdir kallithea-release-XXXXX)

trap cleanup EXIT

echo "Setting up a fresh virtualenv in $venv"

. "$venv/bin/activate"

echo "Install/verify tools needed for building and uploading stuff"

pip install --upgrade -e . -r dev_requirements.txt twine python-ldap python-pam

echo "Cleanup and update copyrights ... and clean checkout"

scripts/run-all-cleanup

scripts/update-copyrights.py

hg up -cr .

echo "Make release build from clean checkout in build/"

rm -rf build dist

hg archive build

cd build

echo "Check that each entry in MANIFEST.in match something"

sed -e 's/[^ ]*[ ]*\([^ ]*\).*/\1/g' MANIFEST.in | xargs ls -lad

echo "Build dist"

python3 setup.py compile_catalog

python3 setup.py sdist

echo "Verify VERSION from kallithea/__init__.py"

namerel=$(cd dist && echo Kallithea-*.tar.gz)

venv=$(mktemp -d kallithea-validatecommits-env-XXXXXX)

resultfile=$(mktemp kallithea-validatecommits-result-XXXXXX)

echo > "$resultfile"

cleanup()

{

    rm -rf /tmp/kallithea-test*

    rm -rf "$venv"

}

finish()

{

    cleanup

    # print (possibly intermediate) results

    cat "$resultfile"

    rm "$resultfile"

}

trap finish EXIT

for rev in $(hg log -r "$1" -T '{node}\n'); do

    hg log -r "$rev"

    hg update "$rev"

    cleanup

    source "$venv/bin/activate"

    pip install --upgrade pip setuptools

    pip install -e . -r dev_requirements.txt python-ldap python-pam

    # run-all-cleanup

    scripts/run-all-cleanup

    if ! hg update --check -q .; then

        echo "run-all-cleanup did not give clean results!"

        result="NOK"

        hg diff

        hg revert -a

    else

        result=" OK"

    fi

    echo "$result: $rev (run-all-cleanup)" >> "$resultfile"

    # pytest

    if py.test; then

        result=" OK"

    else

        result="NOK"

    fi

    echo "$result: $rev (pytest)" >> "$resultfile"

if [ -n "$VIRTUAL_ENV" ]; then

    echo "This script will create its own virtualenv - please don't run it inside an existing one." >&2

    exit 1

fi

cd "$(hg root)"

venv=build/minimum-dependency-versions-venv

log=build/minimum-dependency-versions.log

min_requirements=build/minimum-dependency-versions-requirements.txt

echo "virtualenv: $venv"

echo "log: $log"

echo "minimum requirements file: $min_requirements"

# clean up previous runs

rm -rf "$venv" "$log"

mkdir -p "$venv"

# Make a light weight parsing of setup.py and dev_requirements.txt,

# finding all >= requirements and dumping into a custom requirements.txt

# while fixating the requirement at the lower bound.

sed -n 's/.*"\(.*\)>=\(.*\)".*/\1==\2/p' setup.py > "$min_requirements"

sed 's/>=/==/p' dev_requirements.txt >> "$min_requirements"

source "$venv/bin/activate"

pip install --upgrade pip setuptools

pip install -e . -r "$min_requirements" python-ldap python-pam 2> >(tee "$log" >&2)

# Strip out the known Python 2.7 deprecation message.

sed -i '/DEPRECATION: Python 2\.7 /d' "$log"

# Treat any message on stderr as a problem, for the caller to interpret.

if [ -s "$log" ]; then

    echo

    echo "Error: pip detected following problems:"

    cat "$log"

    echo

    exit 1

fi

freeze_txt=build/minimum-dependency-versions.txt

pip freeze > $freeze_txt

echo "Installation of minimum packages was successful, providing a set of packages as in $freeze_txt . Now running test suite..."

pytest

echo "Test suite execution was successful."

echo "You can now do additional validation using virtual env '$venv'."