.. _upgrade:

===================

Upgrading Kallithea

===================

This describes the process for upgrading Kallithea, independently of the

Kallithea installation method.

.. note::

    If you are upgrading from a RhodeCode installation, you must first

    install Kallithea 0.3.2 and follow the instructions in the 0.3.2

    README to perform a one-time conversion of the database from

    RhodeCode to Kallithea, before upgrading to the latest version

    of Kallithea.

1. Stop the Kallithea web application

-------------------------------------

This step depends entirely on the web server software used to serve

Kallithea, but in any case, Kallithea should not be running during

the upgrade.

.. note::

    If you're using Celery, make sure you stop all instances during the

    upgrade.

2. Create a backup of both database and configuration

-----------------------------------------------------

You are of course strongly recommended to make backups regularly, but it

is *especially* important to make a full database and configuration

backup before performing a Kallithea upgrade.

Back up your configuration

^^^^^^^^^^^^^^^^^^^^^^^^^^

Make a copy of your Kallithea configuration (``.ini``) file.

Back up your database

^^^^^^^^^^^^^^^^^^^^^

If using SQLite, simply make a copy of the Kallithea database (``.db``)

file.

If using PostgreSQL, please consult the documentation for the ``pg_dump``

utility.

If using MariaDB/MySQL, please consult the documentation for the ``mysqldump``

utility.

Look for ``sqlalchemy.url`` in your configuration file to determine

database type, settings, location, etc. If you were running Kallithea 0.3.x or

older, this was ``sqlalchemy.db1.url``.

3. Activate or recreate the Kallithea virtual environment (if any)

------------------------------------------------------------------

.. note::

    If you did not install Kallithea in a virtual environment, skip this step.

For major upgrades, e.g. from 0.3.x to 0.4.x, it is recommended to create a new

virtual environment, rather than reusing the old. For minor upgrades, e.g.

within the 0.4.x range, this is not really necessary (but equally fine).

To create a new virtual environment, please refer to the appropriate

installation page for details. After creating and activating the new virtual

environment, proceed with the rest of the upgrade process starting from the next

section.

To reuse the same virtual environment, first activate it, then verify that you

are using the correct environment by running::

    pip freeze

This will list all packages installed in the current environment. If

Kallithea isn't listed, deactivate the environment and then activate the correct

one, or recreate a new environment. See the appropriate installation page for

details.

4. Install new version of Kallithea

-----------------------------------

Please refer to the instructions for the installation method you

.. _customization:

=============

Customization

=============

There are several ways to customize Kallithea to your needs depending on what

you want to achieve.

HTML/JavaScript/CSS customization

---------------------------------

To customize the look-and-feel of the web interface (for example to add a

company banner or some JavaScript widget or to tweak the CSS style definitions)

you can enter HTML code (possibly with JavaScript and/or CSS) directly via the

*Admin > Settings > Global > HTML/JavaScript customization

block*.

Style sheet customization with Less

-----------------------------------

Kallithea uses `Bootstrap 3`_ and Less_ for its style definitions. If you want

to make some customizations, we recommend to do so by creating a ``theme.less``

file. When you create a file named ``theme.less`` in directory

``kallithea/front-end/`` inside the Kallithea installation, you can use this

file to override the default style. For example, you can use this to override

``@kallithea-theme-main-color``, ``@kallithea-logo-url`` or other `Bootstrap

variables`_.

After creating the ``theme.less`` file, you need to regenerate the CSS files, by

running::

    kallithea-cli front-end-build --no-install-deps

.. _bootstrap 3: https://getbootstrap.com/docs/3.3/

.. _bootstrap variables: https://getbootstrap.com/docs/3.3/customize/#less-variables

.. _less: http://lesscss.org/

indexing and statistics, to add additional code into the push/pull/create/delete

repository hooks (for example to send signals to build bots such as Jenkins) and

even to monkey-patch certain parts of the Kallithea source code (for example

overwrite an entire function, change a global variable, ...).

To generate a skeleton extensions package, run::

    kallithea-cli extensions-create -c my.ini

Behavioral customization: code changes

--------------------------------------

As Kallithea is open-source software, you can make any changes you like directly

in the source code.

We encourage you to send generic improvements back to the

community so that Kallithea can become better. See :ref:`contributing` for more

details.

# -*- coding: utf-8 -*-

# This program is free software: you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program.  If not, see <http://www.gnu.org/licenses/>.

"""

This file was forked by the Kallithea project in July 2014 and later moved.

Original author and date, and relevant copyright and licensing information is below:

:created_on: Mar 6, 2012

:author: marcink

:copyright: (c) 2013 RhodeCode GmbH, and others.

:license: GPLv3, see LICENSE.md for more details.

"""

import os

import click

import pkg_resources

import kallithea

import kallithea.bin.kallithea_cli_base as cli_base

from kallithea.lib.utils2 import ask_ok

@cli_base.register_command(needs_config_file=True)

def extensions_create():

    """Write template file for extending Kallithea in Python.

    """

    here = kallithea.CONFIG['here']

    content = pkg_resources.resource_string(

    )

    if os.path.exists(ext_file):

        msg = ('Extension file %s already exists, do you want '

               'to overwrite it ? [y/n] ') % ext_file

        if not ask_ok(msg):

            click.echo('Nothing done, exiting...')

            return

    dirname = os.path.dirname(ext_file)

    if not os.path.isdir(dirname):

        os.makedirs(dirname)

    with open(ext_file, 'wb') as f:

        f.write(content)

        click.echo('Wrote new extensions file to %s' % ext_file)

:copyright: (c) 2013 RhodeCode GmbH, and others.

:license: GPLv3, see LICENSE.md for more details.

"""

from collections import defaultdict

from pygments import lexers

def get_extension_descriptions():

    """

    Based on what's inside pygments lexers, return a mapping from lowercase

    extensions to lists of very brief descriptions.

    """

    ext_descs = defaultdict(list)

    for lx, t in sorted(lexers.LEXERS.items()):

        desc = lx.replace('Lexer', '')

        for glob in t[-2]:

            s = glob.lstrip('*').lstrip('.').lower()

            start = s.find('[')

            if start > -1 and s.endswith(']'):

                # expand trailing [] range

                prefix = s[:start]

                for char in s[start + 1:-1]:

                    ext_descs[prefix + char].append(desc)

            else:

                # use stripped glob as extension

                ext_descs[s].append(desc)

    return dict(ext_descs)

def get_index_filenames():

    """

    Get list of known indexable filenames from pygment lexer internals

    """

    filenames = []

    for lx, t in sorted(lexers.LEXERS.items()):

        for f in t[-2]:

            if '*' not in f and '[' not in f:

                filenames.append(f)

    return filenames

def get_custom_lexer(extension):

    """

    if there's no custom lexer defined

    """

    import kallithea

    lexer_name = getattr(kallithea.EXTENSIONS, 'EXTRA_LEXERS', {}).get(extension)

    if lexer_name is None:

        return None

    return lexers.get_lexer_by_name(lexer_name)

                    if repo.description != 'unknown'

                    else '%s repository' % name)

            new_repo = repo_model._create_repo(

                repo_name=name,

                repo_type=repo.alias,

                description=desc,

                repo_group=getattr(group, 'group_id', None),

                owner=user,

                enable_downloads=enable_downloads,

                enable_statistics=enable_statistics,

                private=private,

                state=Repository.STATE_CREATED

            )

            sa.commit()

            # we added that repo just now, and make sure it has githook

            # installed, and updated server info

            if new_repo.repo_type == 'git':

                git_repo = new_repo.scm_instance

                ScmModel().install_git_hooks(git_repo)

                # update repository server-info

                log.debug('Running update server info')

                git_repo._update_server_info()

            new_repo.update_changeset_cache()

        elif install_git_hooks:

            if db_repo.repo_type == 'git':

                ScmModel().install_git_hooks(db_repo.scm_instance, force=overwrite_git_hooks)

    removed = []

    # remove from database those repositories that are not in the filesystem

    for repo in sa.query(Repository).all():

        if repo.repo_name not in initial_repo_dict:

            if remove_obsolete:

                log.debug("Removing non-existing repository found in db `%s`",

                          repo.repo_name)

                try:

                    RepoModel().delete(repo, forks='detach', fs_remove=False)

                    sa.commit()

                except Exception:

                    #don't hold further removals on error

                    log.error(traceback.format_exc())

                    sa.rollback()

            removed.append(repo.repo_name)

    return added, removed

def load_extensions(root_path):

    try:

    except FileNotFoundError:

    kallithea.EXTENSIONS = ext

    # Additional mappings that are not present in the pygments lexers

    kallithea.config.conf.LANGUAGES_EXTENSIONS_MAP.update(getattr(ext, 'EXTRA_MAPPINGS', {}))

    # Override any INDEX_EXTENSIONS

    if getattr(ext, 'INDEX_EXTENSIONS', []):

        log.debug('settings custom INDEX_EXTENSIONS')

        kallithea.config.conf.INDEX_EXTENSIONS = getattr(ext, 'INDEX_EXTENSIONS', [])

    # Additional INDEX_EXTENSIONS

    log.debug('adding extra into INDEX_EXTENSIONS')

    kallithea.config.conf.INDEX_EXTENSIONS.extend(getattr(ext, 'EXTRA_INDEX_EXTENSIONS', []))

#==============================================================================

# MISC

#==============================================================================

git_req_ver = StrictVersion('1.7.4')

def check_git_version():

    """

    Checks what version of git is installed on the system, and raise a system exit

    if it's too old for Kallithea to work properly.

    """

    if 'git' not in kallithea.BACKENDS:

        return None

    if not settings.GIT_EXECUTABLE_PATH:

        log.warning('No git executable configured - check "git_path" in the ini file.')

        return None

    try:

        stdout, stderr = GitRepository._run_git_command(['--version'])

    except RepositoryError as e:

        # message will already have been logged as error

        log.warning('No working git executable found - check "git_path" in the ini file.')

        return None

    if stderr:

        log.warning('Error/stderr from "%s --version":\n%s', settings.GIT_EXECUTABLE_PATH, safe_str(stderr))

    if not stdout:

        log.warning('No working git executable found - check "git_path" in the ini file.')

        return None

    output = safe_str(stdout).strip()