When running tests, Kallithea generates a `test.ini` based on template values

in `kallithea/tests/conftest.py` and populates the SQLite database specified

there.

It is possible to avoid recreating the full test database on each invocation of

the tests, thus eliminating the initial delay. To achieve this, run the tests as::

    gearbox serve -c /tmp/kallithea-test-XXX/test.ini --pid-file=test.pid --daemon

    KALLITHEA_WHOOSH_TEST_DISABLE=1 KALLITHEA_NO_TMP_PATH=1 py.test

    kill -9 $(cat test.pid)

In these commands, the following variables are used::

    KALLITHEA_WHOOSH_TEST_DISABLE=1 - skip whoosh index building and tests

    KALLITHEA_NO_TMP_PATH=1 - disable new temp path for tests, used mostly for testing_vcs_operations

You can run individual tests by specifying their path as argument to py.test.

py.test also has many more options, see `py.test -h`. Some useful options

are::

    -k EXPRESSION         only run tests which match the given substring

                          expression. An expression is a python evaluable

                          expression where all names are substring-matched

                          against test names and their parent classes. Example:

    -x, --exitfirst       exit instantly on first error or failed test.

    --lf                  rerun only the tests that failed at the last run (or

                          all if none failed)

    --ff                  run all tests but run the last failures first. This

                          may re-order tests and thus lead to repeated fixture

                          setup/teardown

    --pdb                 start the interactive Python debugger on errors.

    -s, --capture=no      don't capture stdout (any stdout output will be

                          printed immediately)

Performance tests

^^^^^^^^^^^^^^^^^

A number of performance tests are present in the test suite, but they are

not run in a standard test run. These tests are useful to

evaluate the impact of certain code changes with respect to performance.

To run these tests::

    env TEST_PERFORMANCE=1 py.test kallithea/tests/performance

To analyze performance, you could install pytest-profiling_, which enables the

--profile and --profile-svg options to py.test.

.. _pytest-profiling: https://github.com/manahl/pytest-plugins/tree/master/pytest-profiling

.. _contributing-guidelines:

Contribution guidelines

-----------------------

Kallithea is GPLv3 and we assume all contributions are made by the

committer/contributor and under GPLv3 unless explicitly stated. We do care a

lot about preservation of copyright and license information for existing code

that is brought into the project.

Contributions will be accepted in most formats -- such as commits hosted on your

own Kallithea instance, or patches sent by email to the `kallithea-general`_

mailing list.

Make sure to test your changes both manually and with the automatic tests

before posting.

We care about quality and review and keeping a clean repository history. We

might give feedback that requests polishing contributions until they are

"perfect". We might also rebase and collapse and make minor adjustments to your

changes when we apply them.

We try to make sure we have consensus on the direction the project is taking.

Everything non-sensitive should be discussed in public -- preferably on the

mailing list.  We aim at having all non-trivial changes reviewed by at least

one other core developer before pushing. Obvious non-controversial changes will

be handled more casually.

There is a main development branch ("default") which is generally stable so that

it can be (and is) used in production. There is also a "stable" branch that is

almost exclusively reserved for bug fixes or trivial changes. Experimental

changes should live elsewhere (for example in a pull request) until they are

ready.

.. _coding-guidelines:

Coding guidelines

-----------------

We don't have a formal coding/formatting standard. We are currently using a mix

of Mercurial's (https://www.mercurial-scm.org/wiki/CodingStyle), pep8, and

consistency with existing code. Run ``scripts/run-all-cleanup`` before

committing to ensure some basic code formatting consistency.

We try to support the most common modern web browsers. IE9 is still supported

to the extent it is feasible, IE8 is not.

We primarily support Linux and OS X on the server side but Windows should also work.

HTML templates should use 2 spaces for indentation ... but be pragmatic. We

should use templates cleverly and avoid duplication. We should use reasonable

semantic markup with element classes and IDs that can be used for styling and testing.

We should only use inline styles in places where it really is semantic (such as

``display: none``).

JavaScript must use ``;`` between/after statements. Indentation 4 spaces. Inline

multiline functions should be indented two levels -- one for the ``()`` and one for

``{}``.

Variables holding jQuery objects should be named with a leading ``$``.

Commit messages should have a leading short line summarizing the changes. For

bug fixes, put ``(Issue #123)`` at the end of this line.

Use American English grammar and spelling overall. Use `English title case`_ for

page titles, button labels, headers, and 'labels' for fields in forms.

.. _English title case: https://en.wikipedia.org/wiki/Capitalization#Title_case

Template helpers (that is, everything in ``kallithea.lib.helpers``)

should only be referenced from templates. If you need to call a

helper from the Python code, consider moving the function somewhere

else (e.g. to the model).

Notes on the SQLAlchemy session

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Each HTTP request runs inside an independent SQLAlchemy session (as well

as in an independent database transaction). ``Session`` is the session manager

and factory. ``Session()`` will create a new session on-demand or return the

current session for the active thread. Many database operations are methods on

such session instances. The session will generally be removed by

TurboGears automatically.

Database model objects

(almost) always belong to a particular SQLAlchemy session, which means

that SQLAlchemy will ensure that they're kept in sync with the database

(but also means that they cannot be shared across requests).

Objects can be added to the session using ``Session().add``, but this is

rarely needed:

* When creating a database object by calling the constructor directly,

  it must explicitly be added to the session.

* When creating an object using a factory function (like

  ``create_repo``), the returned object has already (by convention)

  been added to the session, and should not be added again.

* When getting an object from the session (via ``Session().query`` or

  any of the utility functions that look up objects in the database),

  it's already part of the session, and should not be added again.

  SQLAlchemy monitors attribute modifications automatically for all

  objects it knows about and syncs them to the database.

SQLAlchemy also flushes changes to the database automatically; manually

calling ``Session().flush`` is usually only necessary when the Python

code needs the database to assign an "auto-increment" primary key ID to

a freshly created model object (before flushing, the ID attribute will

be ``None``).

Debugging

^^^^^^^^^

A good way to trace what Kallithea is doing is to keep an eye on the output on

stdout/stderr of the server process. Perhaps change ``my.ini`` to log at

``DEBUG`` or ``INFO`` level, especially ``[logger_kallithea]``, but perhaps

also other loggers. It is often easier to add additional ``log`` or ``print``

statements than to use a Python debugger.

Sometimes it is simpler to disable ``errorpage.enabled`` and perhaps also

``trace_errors.enable`` to expose raw errors instead of adding extra

processing. Enabling ``debug`` can be helpful for showing and exploring

tracebacks in the browser, but is also insecure and will add extra processing.

TurboGears2 DebugBar

^^^^^^^^^^^^^^^^^^^^

It is possible to enable the TurboGears2-provided DebugBar_, a toolbar overlayed

over the Kallithea web interface, allowing you to see:

* timing information of the current request, including profiling information

* request data, including GET data, POST data, cookies, headers and environment

  variables

* a list of executed database queries, including timing and result values

DebugBar is only activated when ``debug = true`` is set in the configuration

file. This is important, because the DebugBar toolbar will be visible for all

users, and allow them to see information they should not be allowed to see. Like

is anyway the case for ``debug = true``, do not use this in production!

.. _overview:

=====================

Installation overview

=====================

Some overview and some details that can help understanding the options when

installing Kallithea.

1. **Prepare environment and external dependencies.**

    Kallithea needs:

    * A filesystem where the Mercurial and Git repositories can be stored.

    * A database where meta data can be stored.

    * A Python environment where the Kallithea application and its dependencies

      can be installed.

    * A web server that can host the Kallithea web application using the WSGI

      API.

2. **Install Kallithea software.**

    This makes the ``kallithea-cli`` command line tool available.

3. **Prepare front-end files**

    Some front-end files must be fetched or created using ``npm`` and ``node``

    tooling so they can be served to the client as static files.

4. **Create low level configuration file.**

    Use ``kallithea-cli config-create`` to create a ``.ini`` file with database

    connection info, mail server information, configuration for the specified

    web server, etc.

5. **Populate the database.**

    Use ``kallithea-cli db-create`` with the ``.ini`` file to create the

    database schema and insert the most basic information: the location of the

    repository store and an initial local admin user.

6. **Configure the web server.**

    The web server must invoke the WSGI entrypoint for the Kallithea software

    using the ``.ini`` file (and thus the database). This makes the web

    application available so the local admin user can log in and tweak the

    configuration further.

7. **Configure users.**

    The initial admin user can create additional local users, or configure how

    users can be created and authenticated from other user directories.

See the subsequent sections, the separate OS-specific instructions, and

:ref:`setup` for details on these steps.

File system location

--------------------

Kallithea can be installed in many different ways. The main parts are:

- A location for the Kallithea software and its dependencies. This includes

  the Python code, template files, and front-end code. After installation, this

  will be read-only (except when upgrading).

- A location for the ``.ini`` configuration file that tells the Kallithea

  instance which database to use (and thus also the repository location).

  After installation, this will be read-only (except when upgrading).

- A location for various data files and caches for the Kallithea instance. This

  is by default in a ``data`` directory next to the ``.ini`` file. This will

  have to be writable by the running Kallithea service.

- A database. The ``.ini`` file specifies which database to use. The database

  will be a separate service and live elsewhere in the filesystem if using

  PostgreSQL or MariaDB/MySQL. If using SQLite, it will by default live next to

  the ``.ini`` file, as ``kallithea.db``.

- A location for the repositories that are hosted by this Kallithea instance.

  This will have to be writable by the running Kallithea service. The path to

  this location will be configured in the database.

For production setups, one recommendation is to use ``/srv/kallithea`` for the

``.ini`` and ``data``, place the virtualenv in ``venv``, and use a Kallithea

clone in ``kallithea``. Create a ``kallithea`` user, let it own

``/srv/kallithea``, and run as that user when installing.

For simple setups, it is fine to just use something like a ``kallithea`` user

with home in ``/home/kallithea`` and place everything there.

For experiments, it might be convenient to run everything as yourself and work

inside a clone of Kallithea, with the ``.ini`` and SQLite database in the root

of the clone, and a virtualenv in ``venv``.

Python environment

------------------

**Kallithea** is written entirely in Python_ and requires Python version

Given a Python installation, there are different ways of providing the

environment for running Python applications. Each of them pretty much

corresponds to a ``site-packages`` directory somewhere where packages can be

installed.

Kallithea itself can be run from source or be installed, but even when running

from source, there are some dependencies that must be installed in the Python

environment used for running Kallithea.

- Packages *could* be installed in Python's ``site-packages`` directory ... but

  that would require running pip_ as root and it would be hard to uninstall or

  upgrade and is probably not a good idea unless using a package manager.

- Packages could also be installed in ``~/.local`` ... but that is probably

  only a good idea if using a dedicated user per application or instance.

- Finally, it can be installed in a virtualenv. That is a very lightweight

  "container" where each Kallithea instance can get its own dedicated and

  self-contained virtual environment.

We recommend using virtualenv for installing Kallithea.

Locale environment

------------------

In order to ensure a correct functioning of Kallithea with respect to non-ASCII

characters in user names, file paths, commit messages, etc., it is very

important that Kallithea is run with a correct `locale` configuration.

On Unix, environment variables like ``LANG`` or ``LC_ALL`` can specify a language (like

``en_US``) and encoding (like ``UTF-8``) to use for code points outside the ASCII

range. The flexibility of supporting multiple encodings of Unicode has the flip

side of having to specify which encoding to use - especially for Mercurial.

It depends on the OS distribution and system configuration which locales are

available. For example, some Docker containers based on Debian default to only

supporting the ``C`` language, while other Linux environments have ``en_US`` but not

``C``. The ``locale -a`` command will show which values are available on the

current system. Regardless of the actual language, you should normally choose a

locale that has the ``UTF-8`` encoding (note that spellings ``utf8``, ``utf-8``,

``UTF8``, ``UTF-8`` are all referring to the same thing)

For technical reasons, the locale configuration **must** be provided in the

environment in which Kallithea runs - it cannot be specified in the ``.ini`` file.

How to practically do this depends on the web server that is used and the way it

is started. For example, gearbox is often started by a normal user, either

manually or via a script. In this case, the required locale environment

variables can be provided directly in that user's environment or in the script.

However, web servers like Apache are often started at boot via an init script or

service file. Modifying the environment for this case would thus require

root/administrator privileges. Moreover, that environment would dictate the

settings for all web services running under that web server, Kallithea being

just one of them. Specifically in the case of Apache with ``mod_wsgi``, the

locale can be set for a specific service in its ``WSGIDaemonProcess`` directive,

using the ``lang`` parameter.

Installation methods

--------------------

Kallithea must be installed on a server. Kallithea is installed in a Python

environment so it can use packages that are installed there and make itself

available for other packages.

Two different cases will pretty much cover the options for how it can be

installed.

- The Kallithea source repository can be cloned and used -- it is kept stable and

  can be used in production. The Kallithea maintainers use the development

  branch in production. The advantage of installation from source and regularly

  updating it is that you take advantage of the most recent improvements. Using

  it directly from a DVCS also means that it is easy to track local customizations.

  Running ``pip install -e .`` in the source will use pip to install the

  necessary dependencies in the Python environment and create a

  ``.../site-packages/Kallithea.egg-link`` file there that points at the Kallithea

  source.

- Kallithea can also be installed from ready-made packages using a package manager.

  The official released versions are available on PyPI_ and can be downloaded and

  installed with all dependencies using ``pip install kallithea``.

  With this method, Kallithea is installed in the Python environment as any

  other package, usually as a ``.../site-packages/Kallithea-X-py3.8.egg/``

  directory with Python files and everything else that is needed.

  (``pip install kallithea`` from a source tree will do pretty much the same

  but build the Kallithea package itself locally instead of downloading it.)

.. note::

   Kallithea includes front-end code that needs to be processed to prepare

   static files that can be served at run time and used on the client side. The

   tool npm_ is used to download external dependencies and orchestrate the

   processing. The ``npm`` binary must thus be available at install time but is

# -*- coding: utf-8 -*-

# This program is free software: you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program.  If not, see <http://www.gnu.org/licenses/>.

"""

kallithea

~~~~~~~~~

Kallithea, a web based repository management system.

Versioning implementation: http://www.python.org/dev/peps/pep-0386/

This file was forked by the Kallithea project in July 2014.

Original author and date, and relevant copyright and licensing information is below:

:created_on: Apr 9, 2010

:author: marcink

:copyright: (c) 2013 RhodeCode GmbH, (C) 2014 Bradley M. Kuhn, and others.

:license: GPLv3, see LICENSE.md for more details.

"""

import platform

import sys

import celery

VERSION = (0, 7, 0)

BACKENDS = {

    'hg': 'Mercurial repository',

    'git': 'Git repository',

}

CELERY_APP = celery.Celery()  # needed at import time but is lazy and can be configured later

DEFAULT_USER_ID: int  # set by setup_configuration

CONFIG = {}  # set to tg.config when TG app is initialized and calls app_cfg

# URL prefix for non repository related links - must start with `/`

ADMIN_PREFIX = '/_admin'

URL_SEP = '/'

# Linked module for extensions

EXTENSIONS = {}

__version__ = '.'.join(str(each) for each in VERSION)

__platform__ = platform.system()

__license__ = 'GPLv3'

__py_version__ = sys.version_info

__author__ = "Various Authors"

__url__ = 'https://kallithea-scm.org/'

is_windows = __platform__ in ['Windows']

is_unix = not is_windows

#!/usr/bin/env python3

# -*- coding: utf-8 -*-

import os

import platform

import re

import sys

import setuptools

# monkey patch setuptools to use distutils owner/group functionality

from setuptools.command import sdist

here = os.path.abspath(os.path.dirname(__file__))

def _get_meta_var(name, data, callback_handler=None):

    matches = re.compile(r'(?:%s)\s*=\s*(.*)' % name).search(data)

    if matches:

        s = eval(matches.groups()[0])

        if callable(callback_handler):

            return callback_handler(s)

        return s

_meta = open(os.path.join(here, 'kallithea', '__init__.py'), 'r')

_metadata = _meta.read()

_meta.close()

def callback(V):

    return '.'.join(map(str, V[:3])) + '.'.join(V[3:])

__version__ = _get_meta_var('VERSION', _metadata, callback)

__license__ = _get_meta_var('__license__', _metadata)

__author__ = _get_meta_var('__author__', _metadata)

__url__ = _get_meta_var('__url__', _metadata)

# defines current platform

__platform__ = platform.system()

is_windows = __platform__ in ['Windows']

requirements = [

    "alembic >= 1.0.10, < 1.15",

    "gearbox >= 0.1.0, < 1",

    "waitress >= 0.8.8, < 3.1",

    "WebOb >= 1.8, < 1.9",

    "backlash >= 0.1.2, < 1",

    "TurboGears2 >= 2.4, < 2.6",

    "tgext.routes >= 0.2.0, < 1",

    "Beaker >= 1.10.1, < 2",

    "WebHelpers2 >= 2.0, < 2.2",

    "FormEncode >= 1.3.1, < 2.2",

    "SQLAlchemy >= 1.2.9, < 1.4",  # TODO: Upgrade to 1.4 or 2 and fix code

    "Mako >= 0.9.1, < 1.4",

    "Pygments >= 2.2.0, < 2.8",  # TODO: Upgrade and update tests

    "Whoosh >= 2.7.1, < 2.8",

    "celery >= 5, < 5.6",

    "Babel >= 1.3, < 2.18",

    "python-dateutil >= 2.1.0, < 2.10",

    "Markdown >= 2.2.1, < 3.2",  # TODO: Upgrade and update tests

    "docutils >= 0.11, < 0.22",

    "URLObject >= 2.3.4, < 3.1",

    "Routes >= 2.0, < 2.6",

    "dulwich >= 0.19.0, < 0.25",

    "mercurial >= 5.2, < 7.2",

    "decorator >= 4.2.1, < 5.3",

    "Paste >= 2.0.3, < 3.11",

    "bleach >= 3.2, < 5",  # TODO: Upgrade and fix TypeError: clean() got an unexpected keyword argument 'styles'

    "Click >= 7.0, < 8.4",

    "ipaddr >= 2.2.0, < 2.3",

    "paginate >= 0.5, < 0.6",

    "paginate_sqlalchemy >= 0.3.0, < 0.4",

    "bcrypt >= 3.1.0, < 5.1",

    "pip >= 20.0, < 24.1",

    "chardet >= 3",

]

dependency_links = [

]

classifiers = [

    'Development Status :: 4 - Beta',

    'Environment :: Web Environment',

    'Framework :: Pylons',

    'Intended Audience :: Developers',

    'License :: OSI Approved :: GNU General Public License (GPL)',

    'Operating System :: OS Independent',

    'Topic :: Software Development :: Version Control',

]

# additional files from project that goes somewhere in the filesystem

# relative to sys.prefix

data_files = []

description = ('Kallithea is a fast and powerful management tool '

               'for Mercurial and Git with a built in push/pull server, '

               'full text search and code-review.')

keywords = ' '.join([

    'kallithea', 'mercurial', 'git', 'code review',

    'repo groups', 'ldap', 'repository management', 'hgweb replacement',

    'hgwebdir', 'gitweb replacement', 'serving hgweb',

])

# long description

README_FILE = 'README.rst'

try:

    long_description = open(README_FILE).read()

except IOError as err:

    sys.stderr.write(

        "[WARNING] Cannot find file specified as long_description (%s): %s\n"

        % (README_FILE, err)

    )

    long_description = description

sdist_org = sdist.sdist

class sdist_new(sdist_org):

    def initialize_options(self):

        sdist_org.initialize_options(self)

        self.owner = self.group = 'root'

sdist.sdist = sdist_new

packages = setuptools.find_packages(exclude=['ez_setup'])

setuptools.setup(

    name='Kallithea',

    version=__version__,

    description=description,

    long_description=long_description,

    keywords=keywords,

    license=__license__,

    author=__author__,

    author_email='kallithea@sfconservancy.org',

    dependency_links=dependency_links,

    url=__url__,

    install_requires=requirements,

    classifiers=classifiers,

    data_files=data_files,

    packages=packages,

    include_package_data=True,

    message_extractors={'kallithea': [

            ('**.py', 'python', None),

            ('templates/**.mako', 'mako', {'input_encoding': 'utf-8'}),

            ('templates/**.html', 'mako', {'input_encoding': 'utf-8'}),

            ('public/**', 'ignore', None)]},

    zip_safe=False,

    entry_points="""

    [console_scripts]

    kallithea-api =    kallithea.bin.kallithea_api:main

    kallithea-gist =   kallithea.bin.kallithea_gist:main

    kallithea-cli =    kallithea.bin.kallithea_cli:cli

    [paste.app_factory]

    main = kallithea.config.application:make_app

    """,

)