.. _changelog:

Changelog

=========

1.1.1 (**2011-01-06**)

======================

news

----

- added force https option into ini files for easier https usage (no need to

  set server headers with this options)

- small css updates

fixes

-----

- fixed #96 redirect loop on files view on repositories without changesets

- fixed #97 unicode string passed into server header in special cases (mod_wsgi)

  and server crashed with errors

- fixed large tooltips problems on main page

- fixed #92 whoosh indexer is more error proof

1.1.0 (**2010-12-18**)

======================

news

----

- rewrite of internals for vcs >=0.1.10

- uses mercurial 1.7 with dotencode disabled for maintaining compatibility 

  with older clients

- anonymous access, authentication via ldap

- performance upgrade for cached repos list - each repository has it's own 

  cache that's invalidated when needed.

- performance upgrades on repositories with large amount of commits (20K+)

- main page quick filter for filtering repositories

- user dashboards with ability to follow chosen repositories actions

- sends email to admin on new user registration

- added cache/statistics reset options into repository settings

- more detailed action logger (based on hooks) with pushed changesets lists

  and options to disable those hooks from admin panel

- introduced new enhanced changelog for merges that shows more accurate results

- new improved and faster code stats (based on pygments lexers mapping tables, 

  showing up to 10 trending sources for each repository. Additionally stats

  can be disabled in repository settings.

- gui optimizations, fixed application width to 1024px

- added cut off (for large files/changesets) limit into config files

- whoosh, celeryd, upgrade moved to paster command

- other than sqlite database backends can be used

fixes

-----

- fixes #61 forked repo was showing only after cache expired

- fixes #76 no confirmation on user deletes

- fixes #66 Name field misspelled

- fixes #72 block user removal when he owns repositories

- fixes #69 added password confirmation fields

- fixes #87 RhodeCode crashes occasionally on updating repository owner

- fixes #82 broken annotations on files with more than 1 blank line at the end

- a lot of fixes and tweaks for file browser

- fixed detached session issues

- fixed when user had no repos he would see all repos listed in my account

- fixed ui() instance bug when global hgrc settings was loaded for server 

  instance and all hgrc options were merged with our db ui() object

- numerous small bugfixes

(special thanks for TkSoh for detailed feedback)

1.0.2 (**2010-11-12**)

======================

news

----

- tested under python2.7

- bumped sqlalchemy and celery versions

fixes

-----

- fixed #59 missing graph.js

- fixed repo_size crash when repository had broken symlinks

- fixed python2.5 crashes.

1.0.1 (**2010-11-10**)

======================

news

----

- small css updated

fixes

-----

- fixed #53 python2.5 incompatible enumerate calls

- fixed #52 disable mercurial extension for web

- fixed #51 deleting repositories don't delete it's dependent objects

1.0.0 (**2010-11-02**)

======================

- security bugfix simplehg wasn't checking for permissions on commands

  other than pull or push.

- fixed doubled messages after push or pull in admin journal

- templating and css corrections, fixed repo switcher on chrome, updated titles

- admin menu accessible from options menu on repository view

- permissions cached queries

1.0.0rc4  (**2010-10-12**)

==========================

- fixed python2.5 missing simplejson imports (thanks to Jens Bäckman)

- removed cache_manager settings from sqlalchemy meta

.. _installation:

Installation

============

``RhodeCode`` is written entirely in Python, but in order to use it's full

potential there are some third-party requirements. When RhodeCode is used 

together with celery You have to install some kind of message broker,

recommended one is rabbitmq_ to make the async tasks work.

Of course RhodeCode works in sync mode also, then You don't have to install

any third party apps. Celery_ will give You large speed improvement when using

many big repositories. If You plan to use it for 7 or 10 small repositories, it

will work just fine without celery running.

After You decide to Run it with celery make sure You run celeryd using paster

and message broker together with the application.   

Install from Cheese Shop

------------------------

Rhodecode requires python 2.x greater than version 2.5

Easiest way to install ``rhodecode`` is to run::

 easy_install rhodecode

Or::

 pip install rhodecode

If you prefer to install manually simply grab latest release from

   python setup.py install

Step by step installation example

---------------------------------

- Assuming You have installed virtualenv_ create one using. 

  The `--no-site-packages` will make sure non of Your system libs are linked 

  with this virtualenv_  

::

 virtualenv --no-site-packages /var/www/rhodecode-venv

- this will install new virtualenv_ into `/var/www/rhodecode-venv`. 

- Activate the virtualenv_ by running 

::

  source /var/www/rhodecode-venv/bin/activate

- Make a folder for rhodecode somewhere on the filesystem for example 

::

  mkdir /var/www/rhodecode

- Run this command to install rhodecode

::

  easy_install rhodecode 

- this will install rhodecode together with pylons

  and all other required python libraries

Requirements for Celery (optional)

----------------------------------

.. note::

   Installing message broker and using celery is optional, RhodeCode will

   work without them perfectly fine.

**Message Broker** 

- preferred is `RabbitMq <http://www.rabbitmq.com/>`_

- possible other is `Redis <http://code.google.com/p/redis/>`_

For installation instructions You can visit: 

http://ask.github.com/celery/getting-started/index.html

It's very nice tutorial how to start celery_ with rabbitmq_

You can now proceed to :ref:`setup`

-----------------------------------

.. _virtualenv: http://pypi.python.org/pypi/virtualenv  

.. _python: http://www.python.org/

.. _mercurial: http://mercurial.selenic.com/

.. _celery: http://celeryproject.org/

.. _rabbitmq: http://www.rabbitmq.com/

incremental mode.

::

 paster make-index production.ini --repo-location=<location for repos> 

for full index rebuild You can use

::

 paster make-index production.ini -f --repo-location=<location for repos>

- For full text search You can either put crontab entry for

This command can be run even from crontab in order to do periodical 

index builds and keep Your index always up to date. An example entry might 

look like this

::

 /path/to/python/bin/paster /path/to/rhodecode/production.ini --repo-location=<location for repos> 

When using incremental(default) mode whoosh will check last modification date 

of each file and add it to reindex if newer file is available. Also indexing 

daemon checks for removed files and removes them from index. 

Sometime You might want to rebuild index from scratch. You can do that using 

the `-f` flag passed to paster command or, in admin panel You can check 

`build from scratch` flag.

Setting up LDAP support

-----------------------

RhodeCode starting from version 1.1 supports ldap authentication. In order

to use ldap, You have to install python-ldap package. This package is available

via pypi, so You can install it by running

::

 easy_install python-ldap

::

 pip install python-ldap

.. note::

   python-ldap requires some certain libs on Your system, so before installing 

   it check that You have at least `openldap`, and `sasl` libraries.

ldap settings are located in admin->ldap section,

Here's a typical ldap setup::

 Enable ldap  = checked                 #controls if ldap access is enabled

 Host         = host.domain.org         #actual ldap server to connect

 Port         = 389 or 689 for ldaps    #ldap server ports

 Enable LDAPS = unchecked               #enable disable ldaps

 Account      = <account>               #access for ldap server(if required)

 Password     = <password>              #password for ldap server(if required)

 Base DN      = uid=%(user)s,CN=users,DC=host,DC=domain,DC=org

`Account` and `Password` are optional, and used for two-phase ldap 

authentication so those are credentials to access Your ldap, if it doesn't 

support anonymous search/user lookups. 

Base DN must have %(user)s template inside, it's a placer where Your uid used

to login would go, it allows admins to specify not standard schema for uid 

variable

If all data are entered correctly, and `python-ldap` is properly installed

Users should be granted to access RhodeCode wit ldap accounts. When 

logging at the first time an special ldap account is created inside RhodeCode, 

so You can control over permissions even on ldap users. If such user exists 

already in RhodeCode database ldap user with the same username would be not 

able to access RhodeCode.

If You have problems with ldap access and believe You entered correct 

information check out the RhodeCode logs,any error messages sent from 

ldap will be saved there.

Setting Up Celery

-----------------

Since version 1.1 celery is configured by the rhodecode ini configuration files

simply set use_celery=true in the ini file then add / change the configuration 

variables inside the ini file.

Remember that the ini files uses format with '.' not with '_' like celery

so for example setting `BROKER_HOST` in celery means setting `broker.host` in

the config file.

In order to make start using celery run::

 paster celeryd <configfile.ini>

HTTPS support

-------------

There are two ways to enable https, first is to set HTTP_X_URL_SCHEME in

Your http server headers, than rhodecode will recognise this headers and make

proper https redirections, another way is to set `force_https = true` 

in the ini cofiguration to force using https, no headers are needed than to

enable https

Nginx virtual host example

--------------------------

Sample config for nginx using proxy::

 server {

    listen          80;

    server_name     hg.myserver.com;

    access_log      /var/log/nginx/rhodecode.access.log;

    error_log       /var/log/nginx/rhodecode.error.log;

    location / {

            root /var/www/rhodecode/rhodecode/public/;

            if (!-f $request_filename){

                proxy_pass      http://127.0.0.1:5000;

            }

            proxy_set_header X-Url-Scheme $scheme;

            include         /etc/nginx/proxy.conf;  

    }

 }  

Here's the proxy.conf. It's tuned so it'll not timeout on long

pushes and also on large pushes::

    proxy_redirect              off;

    proxy_set_header            Host $host;

    proxy_set_header            X-Host $http_host;

    proxy_set_header            X-Real-IP $remote_addr;

    proxy_set_header            X-Forwarded-For $proxy_add_x_forwarded_for;

    proxy_set_header            Proxy-host $proxy_host;

    client_max_body_size        400m;

    client_body_buffer_size     128k;

    proxy_buffering             off;

    proxy_connect_timeout       3600;

    proxy_send_timeout          3600;

    proxy_read_timeout          3600;

    proxy_buffer_size           8k;

    proxy_buffers               8 32k;

    proxy_busy_buffers_size     64k;

    proxy_temp_file_write_size  64k;

Also when using root path with nginx You might set the static files to false

in production.ini file::

  [app:main]

    use = egg:rhodecode

    full_stack = true

    static_files = false

    lang=en

    cache_dir = %(here)s/data

To not have the statics served by the application. And improve speed.

Apache virtual host example

---------------------------

Sample config for apache using proxy::

    <VirtualHost *:80>

            ServerName hg.myserver.com

            ServerAlias hg.myserver.com

            <Proxy *>

              Order allow,deny

              Allow from all

            </Proxy>

            #important !

            #Directive to properly generate url (clone url) for pylons

            ProxyPreserveHost On

            #rhodecode instance

            ProxyPass / http://127.0.0.1:5000/

            ProxyPassReverse / http://127.0.0.1:5000/

            #to enable https use line below

            #SetEnvIf X-Url-Scheme https HTTPS=1

    </VirtualHost> 

Additional tutorial

http://wiki.pylonshq.com/display/pylonscookbook/Apache+as+a+reverse+proxy+for+Pylons

Apache's example FCGI config

----------------------------

TODO !

Other configuration files

-------------------------

Troubleshooting

---------------

- missing static files ?

 - make sure either to set the `static_files = true` in the .ini file or

   double check the root path for Your http setup. It should point to 

   for example:

   /home/my-virtual-python/lib/python2.6/site-packages/rhodecode/public

- can't install celery/rabbitmq

 - don't worry RhodeCode works without them too. No extra setup required

- long lasting push timeouts ?

 - make sure You set a longer timeouts in Your proxy/fcgi settings, timeouts

   are caused by https server and not RhodeCode

- large pushes timeouts ?

 - make sure You set a proper max_body_size for the http server

.. _virtualenv: http://pypi.python.org/pypi/virtualenv

.. _python: http://www.python.org/

.. _mercurial: http://mercurial.selenic.com/

.. _celery: http://celeryproject.org/

.. _rabbitmq: http://www.rabbitmq.com/

# -*- coding: utf-8 -*-

"""

    rhodecode.controllers.admin.repos

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    Admin controller for RhodeCode

    :created_on: Apr 7, 2010

    :author: marcink

    :license: GPLv3, see COPYING for more details.

"""

# This program is free software; you can redistribute it and/or

# modify it under the terms of the GNU General Public License

# as published by the Free Software Foundation; version 2

# of the License or (at your opinion) any later version of the license.

# 

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

# 

# You should have received a copy of the GNU General Public License

# along with this program; if not, write to the Free Software

# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,

# MA  02110-1301, USA.

import logging

import traceback

import formencode

from operator import itemgetter

from formencode import htmlfill

from paste.httpexceptions import HTTPInternalServerError

from pylons import request, response, session, tmpl_context as c, url

from pylons.controllers.util import abort, redirect

from pylons.i18n.translation import _

from rhodecode.lib import helpers as h

from rhodecode.lib.auth import LoginRequired, HasPermissionAllDecorator, \

    HasPermissionAnyDecorator

from rhodecode.lib.base import BaseController, render

from rhodecode.lib.utils import invalidate_cache, action_logger

from rhodecode.model.db import User

from rhodecode.model.forms import RepoForm

from rhodecode.model.scm import ScmModel

from rhodecode.model.repo import RepoModel

log = logging.getLogger(__name__)

class ReposController(BaseController):

    """REST Controller styled on the Atom Publishing Protocol"""

    # To properly map this controller, ensure your config/routing.py

    # file has a resource setup:

    #     map.resource('repo', 'repos')

    @LoginRequired()

    @HasPermissionAnyDecorator('hg.admin', 'hg.create.repository')

    def __before__(self):

        c.admin_user = session.get('admin_user')

        c.admin_username = session.get('admin_username')

        super(ReposController, self).__before__()

    @HasPermissionAllDecorator('hg.admin')

    def index(self, format='html'):

        """GET /repos: All items in the collection"""

        # url('repos')

        cached_repo_list = ScmModel().get_repos()

        c.repos_list = sorted(cached_repo_list, key=itemgetter('name_sort'))

        return render('admin/repos/repos.html')

    @HasPermissionAnyDecorator('hg.admin', 'hg.create.repository')

    def create(self):

        """POST /repos: Create a new item"""

        # url('repos')

        repo_model = RepoModel()

        _form = RepoForm()()

        form_result = {}

        try:

            form_result = _form.to_python(dict(request.POST))

            repo_model.create(form_result, c.rhodecode_user)

            h.flash(_('created repository %s') % form_result['repo_name'],

                    category='success')

            if request.POST.get('user_created'):

                action_logger(self.rhodecode_user, 'user_created_repo',

                              form_result['repo_name'], '', self.sa)

            else:

                action_logger(self.rhodecode_user, 'admin_created_repo',

                              form_result['repo_name'], '', self.sa)

        except formencode.Invalid, errors:

            c.new_repo = errors.value['repo_name']

            if request.POST.get('user_created'):

                r = render('admin/repos/repo_add_create_repository.html')

            else:

                r = render('admin/repos/repo_add.html')

            return htmlfill.render(

                r,

                defaults=errors.value,

                errors=errors.error_dict or {},

                prefix_error=False,

                encoding="UTF-8")

        if not repo:

            h.flash(_('%s repository is not mapped to db perhaps'

                      ' it was moved or renamed  from the filesystem'

                      ' please run the application again'

                      ' in order to rescan repositories') % repo_name,

                      category='error')

            return redirect(url('repos'))

        try:

            action_logger(self.rhodecode_user, 'admin_deleted_repo',

                              repo_name, '', self.sa)

            repo_model.delete(repo)

            invalidate_cache('get_repo_cached_%s' % repo_name)

            h.flash(_('deleted repository %s') % repo_name, category='success')

        except Exception, e:

            log.error(traceback.format_exc())

            h.flash(_('An error occurred during deletion of %s') % repo_name,

                    category='error')

        return redirect(url('repos'))

    @HasPermissionAllDecorator('hg.admin')

    def delete_perm_user(self, repo_name):

        """

        DELETE an existing repository permission user

        :param repo_name:

        """

        try:

            repo_model = RepoModel()

            repo_model.delete_perm_user(request.POST, repo_name)

        except Exception, e:

            h.flash(_('An error occurred during deletion of repository user'),

                    category='error')

            raise HTTPInternalServerError()

    @HasPermissionAllDecorator('hg.admin')

    def repo_stats(self, repo_name):

        """

        DELETE an existing repository statistics

        :param repo_name:

        """

        try:

            repo_model = RepoModel()

            repo_model.delete_stats(repo_name)

        except Exception, e:

            h.flash(_('An error occurred during deletion of repository stats'),

                    category='error')

        return redirect(url('edit_repo', repo_name=repo_name))

    @HasPermissionAllDecorator('hg.admin')

    def repo_cache(self, repo_name):

        """

        INVALIDATE existing repository cache

        :param repo_name:

        """

        try:

            ScmModel().mark_for_invalidation(repo_name)

        except Exception, e:

            h.flash(_('An error occurred during cache invalidation'),

                    category='error')

        return redirect(url('edit_repo', repo_name=repo_name))

    @HasPermissionAllDecorator('hg.admin')

    def show(self, repo_name, format='html'):

        """GET /repos/repo_name: Show a specific item"""

        # url('repo', repo_name=ID)

    @HasPermissionAllDecorator('hg.admin')

    def edit(self, repo_name, format='html'):

        """GET /repos/repo_name/edit: Form to edit an existing item"""

        # url('edit_repo', repo_name=ID)

        repo_model = RepoModel()

        r = ScmModel().get(repo_name)

        c.repo_info = repo_model.get_by_repo_name(repo_name)

        if c.repo_info is None:

            h.flash(_('%s repository is not mapped to db perhaps'

                      ' it was created or renamed from the filesystem'

                      ' please run the application again'

                      ' in order to rescan repositories') % repo_name,

                      category='error')

            return redirect(url('repos'))

        if c.repo_info.stats:

            last_rev = c.repo_info.stats.stat_on_revision

        else:

            last_rev = 0

        c.stats_revision = last_rev

        c.repo_last_rev = r.revisions[-1] if r.revisions else 0

            c.stats_percentage = 0

        else:

            c.stats_percentage = '%.2f' % ((float((last_rev)) /

                                            c.repo_last_rev) * 100)

        defaults = c.repo_info.get_dict()

        if c.repo_info.user:

            defaults.update({'user':c.repo_info.user.username})

        else:

            replacement_user = self.sa.query(User)\

            .filter(User.admin == True).first().username

            defaults.update({'user':replacement_user})

        c.users_array = repo_model.get_users_js()

        for p in c.repo_info.repo_to_perm:

            defaults.update({'perm_%s' % p.user.username:

                             p.permission.permission_name})

        return htmlfill.render(

            render('admin/repos/repo_edit.html'),

            defaults=defaults,

            encoding="UTF-8",

            force_defaults=False

        )

# -*- coding: utf-8 -*-

"""

    rhodecode.controllers.summary

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    Summary controller for Rhodecode

    :created_on: Apr 18, 2010

    :author: marcink

    :license: GPLv3, see COPYING for more details.

"""

# This program is free software; you can redistribute it and/or

# modify it under the terms of the GNU General Public License

# as published by the Free Software Foundation; version 2

# of the License or (at your opinion) any later version of the license.

# 

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

# 

# You should have received a copy of the GNU General Public License

# along with this program; if not, write to the Free Software

# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,

# MA  02110-1301, USA.

import calendar

import logging

from time import mktime

from vcs.exceptions import ChangesetError

from pylons import tmpl_context as c, request, url

from pylons.i18n.translation import _

from rhodecode.model.scm import ScmModel

from rhodecode.model.db import Statistics

from rhodecode.lib.auth import LoginRequired, HasRepoPermissionAnyDecorator

from rhodecode.lib.base import BaseController, render

from rhodecode.lib.utils import OrderedDict, EmptyChangeset

from rhodecode.lib.celerylib import run_task

from rhodecode.lib.celerylib.tasks import get_commits_stats

from webhelpers.paginate import Page

try:

    import json

except ImportError:

    #python 2.5 compatibility

    import simplejson as json

log = logging.getLogger(__name__)

class SummaryController(BaseController):

    @LoginRequired()

    @HasRepoPermissionAnyDecorator('repository.read', 'repository.write',

                                   'repository.admin')

    def __before__(self):

        super(SummaryController, self).__before__()

    def index(self):

        scm_model = ScmModel()

        c.repo_info = scm_model.get_repo(c.repo_name)

        c.following = scm_model.is_following_repo(c.repo_name,

                                             c.rhodecode_user.user_id)

        def url_generator(**kw):

            return url('shortlog_home', repo_name=c.repo_name, **kw)

        c.repo_changesets = Page(c.repo_info, page=1, items_per_page=10,

                                 url=url_generator)

        e = request.environ

        if self.rhodecode_user.username == 'default':

            password = ':default'

        else:

            password = ''

        uri = u'%(protocol)s://%(user)s%(password)s@%(host)s%(prefix)s/%(repo_name)s' % {

                                        'protocol': e.get('wsgi.url_scheme'),

                                        'user':str(c.rhodecode_user.username),

                                        'password':password,

                                        'host':e.get('HTTP_HOST'),

                                        'prefix':e.get('SCRIPT_NAME'),

                                        'repo_name':c.repo_name, }

        c.clone_repo_url = uri

        c.repo_tags = OrderedDict()

        for name, hash in c.repo_info.tags.items()[:10]:

            try:

                c.repo_tags[name] = c.repo_info.get_changeset(hash)

            except ChangesetError:

                c.repo_tags[name] = EmptyChangeset(hash)

        c.repo_branches = OrderedDict()

        for name, hash in c.repo_info.branches.items()[:10]:

            try:

                c.repo_branches[name] = c.repo_info.get_changeset(hash)

            except ChangesetError:

                c.repo_branches[name] = EmptyChangeset(hash)

        if c.repo_info.dbrepo.enable_statistics:

            c.no_data_msg = _('No data loaded yet')

            run_task(get_commits_stats, c.repo_info.name, ts_min_y, ts_max_y)

        else:

            c.no_data_msg = _('Statistics update are disabled for this repository')

        c.ts_min = ts_min_m

        c.ts_max = ts_max_y

        stats = self.sa.query(Statistics)\

            .filter(Statistics.repository == c.repo_info.dbrepo)\

            .scalar()

        if stats and stats.languages:

            c.no_data = False is c.repo_info.dbrepo.enable_statistics