================

Kallithea README

================

About

-----

**Kallithea** is a fast and powerful management tool for Mercurial_ and Git_

with a built-in push/pull server, full text search and code-review. It works on

http/https and has a built in permission/authentication system with the ability

to authenticate via LDAP or ActiveDirectory. Kallithea also provides simple API

so it's easy to integrate with existing external systems.

Kallithea is similar in some respects to GitHub_ or Bitbucket_, however

Kallithea can be run as standalone hosted application on your own server. It is

open-source donationware and focuses more on providing a customised,

self-administered interface for Mercurial_ and Git_ repositories. Kallithea

works on Unix-like systems and Windows, and is powered by the vcs_ library

created by Łukasz Balcerzak and Marcin Kuźmiński to uniformly handle multiple

version control systems.

Kallithea was forked from RhodeCode in July 2014 and has been heavily modified.

Installation

------------

Kallithea requires Python_ 2.x and it is recommended to install it in a

virtualenv_. Official releases of Kallithea can be installed with::

    pip install kallithea

The development repository is kept very stable and used in production by the

developers - you can do the same.

Please visit https://docs.kallithea-scm.org/en/latest/installation.html for

more details.

Source code

-----------

The latest sources can be obtained from

https://kallithea-scm.org/repos/kallithea.

The issue tracker and a repository mirror can be found at Bitbucket_ on

https://bitbucket.org/conservancy/kallithea.

------------------

- Has its own middleware to handle Mercurial_ and Git_ protocol requests. Each

  request is authenticated and logged together with IP address.

- Built for speed and performance. You can make multiple pulls/pushes

  simultaneously. Proven to work with thousands of repositories and users.

- Supports http/https, LDAP, AD, proxy-pass authentication.

- Full permissions (private/read/write/admin) together with IP restrictions for

  each repository, additional explicit forking, repositories group and

  repository creation permissions.

- User groups for easier permission management.

- Repository groups let you group repos and manage them easier. They come with

  permission delegation features, so you can delegate groups management.

- Users can fork other users repos, and compare them at any time.

- Built-in versioned paste functionality (Gist) for sharing code snippets.

- Integrates easily with other systems, with custom created mappers you can

  connect it to almost any issue tracker, and with a JSON-RPC API you can make

  much more.

- Built-in commit API lets you add, edit and commit files right from Kallithea

  web interface using simple editor or upload binary files using simple form.

- Powerful pull request driven review system with inline commenting, changeset

  statuses, and notification system.

- Importing and syncing repositories from remote locations for Git_, Mercurial_

  and Subversion.

- Mako templates let you customize the look and feel of the application.

- Beautiful diffs, annotations and source code browsing all colored by

  pygments. Raw diffs are made in Git-diff format for both VCS systems,

  including Git_ binary-patches.

- Mercurial_ and Git_ DAG graphs and Flot-powered graphs with zooming and

  statistics to track activity for repositories.

- Admin interface with user/permission management. Admin activity journal, logs

  pulls, pushes, forks, registrations and other actions made by all users.

- Server side forks. It is possible to fork a project and modify it freely

  without breaking the main repository.

- reST and Markdown README support for repositories.

- Full text search powered by Whoosh on the source files, commit messages, and

  file names. Built-in indexing daemons, with optional incremental index build

  (no external search servers required all in one application).

- Setup project descriptions/tags and info inside built in DB for easy,

  non-filesystem operations.

- Intelligent cache with invalidation after push or project change, provides

  high performance and always up to date data.

- RSS/Atom feeds, Gravatar support, downloadable sources as zip/tar/gz.

- Optional async tasks for speed and performance using Celery_.

- Backup scripts can do backup of whole app and send it over scp to desired

  location.

- Based on Pylons, SQLAlchemy, SQLite, Whoosh, vcs.

License

-------

**Kallithea** is released under the GPLv3 license. Kallithea is a `Software

Freedom Conservancy`_ project and thus controlled by a non-profit organization.

No commercial entity can take ownership of the project and change the

direction.

Kallithea started out as an effort to make sure the existing GPLv3 codebase

would stay available under a legal license. Kallithea thus has to stay GPLv3

compatible ... but we are also happy it is GPLv3 and happy to keep it that way.

A different license (such as AGPL) could perhaps help attract a different

community with a different mix of Free Software people and companies but we are

happy with the current focus.

Community

---------

**Kallithea** is maintained by its users who contribute the fixes they would

 like to see.

Get in touch with the rest of the community:

- Join the mailing list users and developers - see

  http://lists.sfconservancy.org/mailman/listinfo/kallithea-general.

- Use IRC and join #kallithea on FreeNode (irc.freenode.net) or use

  http://webchat.freenode.net/?channels=kallithea.

- Follow Kallithea on Twitter, **@KallitheaSCM**.

- Issues can be reported at `issue tracker

  <https://bitbucket.org/conservancy/kallithea/issues>`_.

   .. note::

       Please try to read the documentation before posting any issues,

       especially the **troubleshooting section**

Online documentation

--------------------

Online documentation for the current version of Kallithea is available at

https://pythonhosted.org/Kallithea/. Documentation for the current development

version can be found on https://docs.kallithea-scm.org/.

You can also build the documentation locally: go to ``docs/`` and run::

   make html

.. note:: You need to have Sphinx_ installed to build the

          documentation. If you don't have Sphinx_ installed you can

          install it via the command: ``pip install sphinx`` .

Converting from RhodeCode

-------------------------

Currently, you have two options for working with an existing RhodeCode

database:

- keep the database unconverted (intended for testing and evaluation)

- convert the database in a one-time step

~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Interoperability with RhodeCode 2.2.X installations is provided so you don't

have to immediately commit to switching to Kallithea. This option will most

likely go away once the two projects have diverged significantly.

To run Kallithea on a RhodeCode database, run::

   echo "BRAND = 'rhodecode'" > kallithea/brand.py

This location will depend on where you installed Kallithea. If you installed

via::

   python setup.py install

then you will find this location at

``$VIRTUAL_ENV/lib/python2.7/site-packages/Kallithea-0.1-py2.7.egg/kallithea``.

~~~~~~~~~~~~~~~~~~~

Alternatively, if you would like to convert the database for good, you can use

a helper script provided by Kallithea. This script will operate directly on the

database, using the database string you can find in your ``production.ini`` (or

``development.ini``) file. For example, if using SQLite::

   cd /path/to/kallithea

   cp /path/to/rhodecode/rhodecode.db kallithea.db

   pip install sqlalchemy-migrate

   python kallithea/bin/rebranddb.py sqlite:///kallithea.db

.. Note::

   If you started out using the branding interoperability approach mentioned

   above, watch out for stray brand.pyc after removing brand.py.

.. _virtualenv: http://pypi.python.org/pypi/virtualenv

.. _Python: http://www.python.org/

.. _Sphinx: http://sphinx.pocoo.org/

.. _Mercurial: http://mercurial.selenic.com/

.. _Bitbucket: http://bitbucket.org/

.. _GitHub: http://github.com/

.. _Subversion: http://subversion.tigris.org/

.. _Git: http://git-scm.com/

.. _Celery: http://celeryproject.org/

.. _vcs: http://pypi.python.org/pypi/vcs

.. _Software Freedom Conservancy: http://sfconservancy.org/

.. _models:

========================

========================

.. automodule:: kallithea.model

   :members:

.. automodule:: kallithea.model.comment

   :members:

.. automodule:: kallithea.model.notification

   :members:

.. automodule:: kallithea.model.permission

   :members:

.. automodule:: kallithea.model.repo_permission

   :members:

.. automodule:: kallithea.model.repo

   :members:

.. automodule:: kallithea.model.repo_group

   :members:

.. automodule:: kallithea.model.scm

   :members:

.. automodule:: kallithea.model.user

   :members:

.. automodule:: kallithea.model.user_group

   :members:

.. _index:

Kallithea Documentation

**Readme**

.. toctree::

   :maxdepth: 1

   readme

**Installation**

.. toctree::

   :maxdepth: 1

   overview

   installation

   installation_win

   installation_win_old

   installation_iis

   setup

**Usage**

.. toctree::

   :maxdepth: 1

   usage/general

   usage/vcs_support

   usage/locking

   usage/statistics

.. toctree::

   :maxdepth: 1

   usage/email

   usage/performance

   usage/backup

   usage/debugging

   usage/troubleshooting

.. toctree::

   :maxdepth: 1

   contributing

   changelog

**API**

.. toctree::

   :maxdepth: 1

   api/api

   api/models

Other topics

------------

* :ref:`genindex`

* :ref:`search`

.. _virtualenv: http://pypi.python.org/pypi/virtualenv

.. _python: http://www.python.org/

.. _django: http://www.djangoproject.com/

.. _mercurial: http://mercurial.selenic.com/

.. _bitbucket: http://bitbucket.org/

.. _subversion: http://subversion.tigris.org/

.. _git: http://git-scm.com/

.. _celery: http://celeryproject.org/

.. _Sphinx: http://sphinx.pocoo.org/

.. _vcs: http://pypi.python.org/pypi/vcs

.. _installation:

==========================

Installation on Unix/Linux

==========================

Here are more details about 3 ways to install Kallithea:

- :ref:`installation-source`: The simplest way to keep the installation

  uptodate and keep track of local customizations is to run directly from

  source in a Kallithea repository clone and use virtualenv.

- :ref:`installation-virtualenv`: If you prefer to only use released versions

  of Kallithea, the recommended method is to install Kallithea in a virtual

  Python environment using `virtualenv`. The advantages of this method over

  direct installation is that Kallithea and its dependencies are completely

  contained inside the virtualenv (which also means you can have multiple

  installations side by side or remove it entirely by just removing the

  virtualenv directory) and does not require root privileges.

- :ref:`installation-without-virtualenv`: The alternative method of installing

  a Kallithea release is using standard pip. The package will be installed in

  the same location as all other Python packages you have ever installed. As a

  result, removing it is not as straightforward as with a virtualenv, as you'd

  have to remove its dependencies manually and make sure that they are not

  needed by other packages.

.. _installation-source:

Installation from repository source

-----------------------------------

To install Kallithea in a virtualenv using the stable branch of the development

repository, follow the instructions below::

        hg clone https://kallithea-scm.org/repos/kallithea -u stable

        cd kallithea

        virtualenv ../kallithea-venv

        source ../kallithea-venv/bin/activate

        python setup.py develop

        python setup.py compile_catalog   # for translation of the UI

You can now proceed to :ref:`setup`.

To upgrade, simply update the repository with ``hg pull -u`` and restart the

server.

.. _installation-virtualenv:

Installing a released version in a virtualenv

---------------------------------------------

It is highly recommended to use a separate virtualenv_ for installing Kallithea.

This way, all libraries required by Kallithea will be installed separately from your

main Python installation and other applications and things will be less

problematic when upgrading the system or Kallithea.

An additional benefit of virtualenv_ is that it doesn't require root privileges.

- Assuming you have installed virtualenv_, create a new virtual environment

  for example, in `/srv/kallithea/venv`, using the virtualenv command::

    virtualenv /srv/kallithea/venv

- Activate the virtualenv_ in your current shell session by running::

    source /srv/kallithea/venv/bin/activate

.. note:: You can't use UNIX ``sudo`` to source the ``virtualenv`` script; it

   will "activate" a shell that terminates immediately. It is also perfectly

   acceptable (and desirable) to create a virtualenv as a normal user.

- Make a folder for Kallithea data files, and configuration somewhere on the

  filesystem. For example::

    mkdir /srv/kallithea

- Go into the created directory and run this command to install Kallithea::

    pip install kallithea

  Alternatively, download a .tar.gz from http://pypi.python.org/pypi/Kallithea,

  extract it and run::

    python setup.py install

- This will install Kallithea together with pylons_ and all other required

  python libraries into the activated virtualenv.

You can now proceed to :ref:`setup`.

.. _installation-without-virtualenv:

Installing a released version without virtualenv

------------------------------------------------

For installation without virtualenv, 'just' use::

    pip install kallithea

Note that this method requires root privileges and will install packages

globally without using the system's package manager.

To install as a regular user in ``~/.local``, you can use::

    pip install --user kallithea

You can now proceed to :ref:`setup`.

Upgrading Kallithea from Python Package Index (PyPI)

.. note::

   It is strongly recommended that you **always** perform a database and

   configuration backup before doing an upgrade.

   These directions will use '{version}' to note that this is the version of

   Kallithea that these files were used with.  If backing up your Kallithea

   instance from version 0.1 to 0.2, the ``my.ini`` file could be

   backed up to ``my.ini.0-1``.

If using a SQLite database, stop the Kallithea process/daemon/service, and

then make a copy of the database file::

 service kallithea stop

 cp kallithea.db kallithea.db.{version}

Back up your configuration file::

 cp my.ini my.ini.{version}

Ensure that you are using the Python virtual environment that you originally

installed Kallithea in by running::

 pip freeze

This will list all packages installed in the current environment.  If

Kallithea isn't listed, activate the correct virtual environment::

 source /srv/kallithea/venv/bin/activate

Once you have verified the environment you can upgrade Kallithea with::

 pip install --upgrade kallithea

Then run the following command from the installation directory::

 paster make-config Kallithea my.ini

This will display any changes made by the new version of Kallithea to your

current configuration. It will try to perform an automerge. It is recommended

that you recheck the content after the automerge.

.. note::

   Please always make sure your .ini files are up to date. Errors can

   often be caused by missing parameters added in new versions.

It is also recommended that you rebuild the whoosh index after upgrading since

the new whoosh version could introduce some incompatible index changes. Please

read the changelog to see if there were any changes to whoosh.

The final step is to upgrade the database. To do this simply run::

 paster upgrade-db my.ini

This will upgrade the schema and update some of the defaults in the database,

and will always recheck the settings of the application, if there are no new

options that need to be set.

.. note::

   The DB schema upgrade library has some limitations and can sometimes fail if you try to

   upgrade from older major releases. In such a case simply run upgrades sequentially, e.g.,

   upgrading from 0.1.X to 0.3.X should be done like this: 0.1.X. > 0.2.X > 0.3.X

   You can always specify what version of Kallithea you want to install for example in pip

   `pip install Kallithea==0.2`

You may find it helpful to clear out your log file so that new errors are

readily apparent::

 echo > kallithea.log

Once that is complete, you may now start your upgraded Kallithea Instance::

 service kallithea start

Or::

 paster serve /srv/kallithea/my.ini

.. note::

   If you're using Celery, make sure you restart all instances of it after

   upgrade.

.. _virtualenv: http://pypi.python.org/pypi/virtualenv

.. _pylons: http://www.pylonsproject.org/

.. _installation_iis:

Installing Kallithea on Microsoft Internet Information Services (IIS)

=====================================================================

The following is documented using IIS 7/8 terminology. There should be nothing

preventing you from applying this on IIS 6 well.

.. note::

    For the best security, it is strongly recommended to only host the site over

    a secure connection, e.g. using TLS.

Prerequisites

-------------

Apart from the normal requirements for Kallithea, it is also necessary to get an

ISAPI-WSGI bridge module, e.g. isapi-wsgi.

Installation

------------

The following will assume that your Kallithea is at ``c:\inetpub\kallithea`` and

will be served from the root of its own website. The changes to serve it in its

own virtual folder will be noted where appropriate.

................

Make sure that there is a unique application pool for the Kallithea application

with an identity that has read access to the Kallithea distribution.

The application pool does not need to be able to run any managed code. If you

are using a 32-bit Python installation, then you must enable 32 bit program in

the advanced settings for the application pool otherwise Python will not be able

to run on the website and consequently, Kallithea will not be able to run.

.. note::

    The application pool can be the same as an existing application pool as long

    as the requirements to Kallithea are enabled by the existing application

    pool.

.............

The ISAPI handler can be generated using::

    paster install-iis my.ini --root=/

This will generate a ``dispatch.py`` file in the current directory that contains

the necessary components to finalize an installation into IIS. Once this file

has been generated, it is necessary to run the following command due to the way

that ISAPI-WSGI is made::

    python dispatch.py install

This accomplishes two things: generating an ISAPI compliant DLL file,

``_dispatch.dll``, and installing a script map handler into IIS for the

``--root`` specified above pointing to ``_dispatch.dll``.

The ISAPI handler is registered to all file extensions, so it will automatically

be the one handling all requests to the specified root. When the website starts

the ISAPI handler, it will start a thread pool managed wrapper around the paster

middleware WSGI handler that Kallithea runs within and each HTTP request to the

site will be processed through this logic henceforth.

Authentication with Kallithea using IIS authentication modules

..............................................................

The recommended way to handle authentication with Kallithea using IIS is to let

IIS handle all the authentication and just pass it to Kallithea.

To move responsibility into IIS from Kallithea, we need to configure Kallithea

to let external systems handle authentication and then let Kallithea create the

user automatically. To do this, access the administration's authentication page

and enable the ``kallithea.lib.auth_modules.auth_container`` plugin. Once it is

added, enable it with the ``REMOTE_USER`` header and check *Clean username*.

Finally, save the changes on this page.

Switch to the administration's permissions page and disable anonymous access,

otherwise Kallithea will not attempt to use the authenticated user name. By

default, Kallithea will populate the list of users lazily as they log in. Either

disable external auth account activation and ensure that you pre-populate the

user database with an external tool, or set it to *Automatic activation of

external account*. Finally, save the changes.

The last necessary step is to enable the relevant authentication in IIS, e.g.

Windows authentication.

Troubleshooting

---------------

Typically, any issues in this setup will either be entirely in IIS or entirely

in Kallithea (or Kallithea's WSGI/paster middleware). Consequently, two

different options for finding issues exist: IIS' failed request tracking which

is great at finding issues until they exist inside Kallithea, at which point the

ISAPI-WSGI wrapper above uses ``win32traceutil``, which is part of ``pywin32``.

In order to dump output from WSGI using ``win32traceutil`` it is sufficient to

type the following in a console window::

    python -m win32traceutil

and any exceptions occurring in the WSGI layer and below (i.e. in the Kallithea

application itself) that are uncaught, will be printed here complete with stack

traces, making it a lot easier to identify issues.

.. _installation_win:

Installation and upgrade on Windows (7/Server 2008 R2 and newer)

================================================================

First time install

::::::::::::::::::

Target OS: Windows 7 and newer or Windows Server 2008 R2 and newer

Tested on Windows 8.1, Windows Server 2008 R2 and Windows Server 2012

To install on an older version of Windows, see `<installation_win_old.html>`_

Step 1 - Install Python

-----------------------

Install Python 2.x.y (x = 6 or 7). Latest version is recommended. If you need another version, they can run side by side.

.. warning:: Python 3.x is not supported.

- Download Python 2.x.y from http://www.python.org/download/

- Choose and click on the version

- Click on "Windows X86-64 Installer" for x64 or "Windows x86 MSI installer" for Win32.

- Disable UAC or run the installer with admin privileges. If you chose to disable UAC, do not forget to reboot afterwards.

While writing this guide, the latest version was v2.7.9.

Remember the specific major and minor versions installed, because they will

be needed in the next step. In this case, it is "2.7".

Step 2 - Python BIN

-------------------

Add Python BIN folder to the path. This can be done manually (editing

"PATH" environment variable) or by using Windows Support Tools that

come pre-installed in Windows Vista/7 and later.

Open a CMD and type::

  SETX PATH "%PATH%;[your-python-path]" /M

Please substitute [your-python-path] with your Python installation

path. Typically this is ``C:\\Python27``.

Step 3 - Install pywin32 extensions

-----------------------------------

Download pywin32 from:

http://sourceforge.net/projects/pywin32/files/

- Click on "pywin32" folder

- Click on the first folder (in this case, Build 219, maybe newer when you try)

- Choose the file ending with ".amd64-py2.x.exe" (".win32-py2.x.exe"

  for Win32) where x is the minor version of Python you installed.

  When writing this guide, the file was:

  http://sourceforge.net/projects/pywin32/files/pywin32/Build%20219/pywin32-219.win-amd64-py2.7.exe/download

  (x64)

  http://sourceforge.net/projects/pywin32/files/pywin32/Build%20219/pywin32-219.win32-py2.7.exe/download

  (Win32)

Step 4 - Install pip

--------------------

pip is a package management system for Python. You will need it to install Kallithea and its dependencies.

If you installed Python 2.7.9+, you already have it (as long as you ran the installer with admin privileges or disabled UAC).

If it was not installed or if you are using Python>=2.6,<2.7.9:

- Go to https://bootstrap.pypa.io

- Right-click on get-pip.py and choose Saves as...

- Run "python get-pip.py" in the folder where you downloaded get-pip.py (may require admin access).

.. note::

   See http://stackoverflow.com/questions/4750806/how-to-install-pip-on-windows

   for details and alternative methods.

Note that pip.exe will be placed inside your Python installation's

Scripts folder, which is likely not on your path. To correct this,

open a CMD and type::

  SETX PATH "%PATH%;[your-python-path]\Scripts" /M

-----------------------------------

Create a Kallithea folder structure.

This is only an example to install Kallithea. Of course, you can

change it. However, this guide will follow the proposed structure, so

please later adapt the paths if you change them. Folders without

spaces are recommended.

Create the following folder structure::

  C:\Kallithea

  C:\Kallithea\Bin

  C:\Kallithea\Env

  C:\Kallithea\Repos

Step 6 - Install virtualenv

---------------------------

.. note::

   A python virtual environment will allow for isolation between the Python packages of your system and those used for Kallithea.

   It is strongly recommended to use it to ensure that Kallithea does not change a dependency that other software uses or vice versa.

In a command prompt type::

  pip install virtualenv

Virtualenv will now be inside your Python Scripts path (C:\\Python27\\Scripts or similar).

To create a virtual environment, run::

  virtualenv C:\Kallithea\Env

Step 7 - Install Kallithea

--------------------------

In order to install Kallithea, you need to be able to run "pip install kallithea". It will use pip to install the Kallithea Python package and its dependencies.

Some Python packages use managed code and need to be compiled.

This can be done on Linux without any special steps. On Windows, you will need to install Microsoft Visual C++ compiler for Python 2.7.

Download and install "Microsoft Visual C++ Compiler for Python 2.7" from http://aka.ms/vcpython27

.. note::

  You can also install the dependencies using already compiled Windows binaries packages. A good source of compiled Python packages is http://www.lfd.uci.edu/~gohlke/pythonlibs/. However, not all of the necessary packages for Kallithea are on this site and some are hard to find, so we will stick with using the compiler.

In a command prompt type (adapting paths if necessary)::

  cd C:\Kallithea\Env\Scripts

  activate

The prompt will change into "(Env) C:\\Kallithea\\Env\\Scripts" or similar

(depending of your folder structure). Then type::

  pip install kallithea

.. note:: This will take some time. Please wait patiently until it is fully

          complete. Some warnings will appear. Don't worry, they are

          normal.

-------------------------------

Mercurial being a python package, it was installed automatically when doing "pip install kallithea".

You need to install git manually if you want Kallithea to be able to host git repositories.

See http://git-scm.com/book/en/v2/Getting-Started-Installing-Git#Installing-on-Windows for instructions.

Step 9 - Configuring Kallithea

------------------------------

Steps taken from `<setup.html>`_

You have to use the same command prompt as in Step 7, so if you closed

it, reopen it following the same commands (including the "activate"

one). When ready, type::

  cd C:\Kallithea\Bin

  paster make-config Kallithea production.ini

Then you must edit production.ini to fit your needs (IP address, IP

port, mail settings, database, etc.). `NotePad++`__ or a similar text

editor is recommended to properly handle the newline character

differences between Unix and Windows.

__ http://notepad-plus-plus.org/

For the sake of simplicity, run it with the default settings. After your edits (if any) in the previous command prompt, type::

  paster setup-db production.ini

.. warning:: This time a *new* database will be installed. You must

             follow a different step to later *upgrade* to a newer

             Kallithea version)

The script will ask you for confirmation about creating a new database, answer yes (y)

The script will ask you for the repository path, answer C:\\Kallithea\\Repos (or similar).

The script will ask you for the admin username and password, answer "admin" + "123456" (or whatever you want)

The script will ask you for admin mail, answer "admin@xxxx.com" (or whatever you want).