.. _changelog:

=========

Changelog

=========

1.4.0 (**2012-XX-XX**)

----------------------

:status: in-progress

:branch: beta

news

++++

- new codereview system

- email map, allowing users to have multiple email addresses mapped into

  their accounts

- improved git-hook system. Now all actions for git are logged into journal

  including pushed revisions, user and IP address

- changed setup-app into setup-rhodecode and added default options to it.

- new git repos are created as bare now by default

- #464 added links to groups in permission box

- #465 mentions autocomplete inside comments boxes

- #469 added --update-only option to whoosh to re-index only given list

  of repos in index 

- rhodecode-api CLI client

- new git http protocol replaced buggy dulwich implementation.

  Now based on pygrack & gitweb

- Improved RSS/ATOM feeds. Discoverable by browsers using proper headers, and 

  reformated based on user suggestions. Additional rss/atom feeds for user

  journal

- various i18n improvements

- #478 permissions overview for admin in user edit view

- File view now displays small gravatars off all authors of given file

- Implemented landing revisions. Each repository will get landing_rev attribute

  that defines 'default' revision/branch for generating readme files

- Implemented #509, RhodeCode enforces SSL for push/pulling if requested.

- Import remote svn repositories to mercurial using hgsubversion  

- Fixed #508 RhodeCode now has a option to explicitly set forking permissions

- RhodeCode can use alternative server for generating avatar icons

fixes

+++++

- improved translations

- fixes issue #455 Creating an archive generates an exception on Windows

- fixes #448 Download ZIP archive keeps file in /tmp open and results 

  in out of disk space

- fixes issue #454 Search results under Windows include proceeding

  backslash

- fixed issue #450. Rhodecode no longer will crash when bad revision is

  present in journal data.

- fix for issue #417, git execution was broken on windows for certain

  commands.

- fixed #413. Don't disable .git directory for bare repos on deleting

- fixed issue #459. Changed the way of obtaining logger in reindex task.

- fixed #453 added ID field in whoosh SCHEMA that solves the issue of

  reindexing modified files

- fixed #481 rhodecode emails are sent without Date header 

- fixed #458 wrong count when no repos are present

- fixed issue #492 missing `\ No newline at end of file` test at the end of 

  new chunk in html diff

- full text search now works also for commit messages

1.3.6 (**2012-05-17**)

----------------------

news

++++

- chinese traditional translation

- changed setup-app into setup-rhodecode and added arguments for auto-setup 

  mode that doesn't need user interaction 

fixes

+++++

- fixed no scm found warning

- fixed __future__ import error on rcextensions

- made simplejson required lib for speedup on JSON encoding

- fixes #449 bad regex could get more than revisions from parsing history

- don't clear DB session when CELERY_EAGER is turned ON

1.3.5 (**2012-05-10**)

----------------------

news

++++

    return safe_str

def engine_from_config(configuration, prefix='sqlalchemy.', **kwargs):

    """

    Custom engine_from_config functions that makes sure we use NullPool for

    file based sqlite databases. This prevents errors on sqlite. This only

    applies to sqlalchemy versions < 0.7.0

    """

    import sqlalchemy

    from sqlalchemy import engine_from_config as efc

    import logging

    if int(sqlalchemy.__version__.split('.')[1]) < 7:

        # This solution should work for sqlalchemy < 0.7.0, and should use

        # proxy=TimerProxy() for execution time profiling

        from sqlalchemy.pool import NullPool

        url = configuration[prefix + 'url']

        if url.startswith('sqlite'):

            kwargs.update({'poolclass': NullPool})

        return efc(configuration, prefix, **kwargs)

    else:

        import time

        from sqlalchemy import event

        from sqlalchemy.engine import Engine

        log = logging.getLogger('sqlalchemy.engine')

        BLACK, RED, GREEN, YELLOW, BLUE, MAGENTA, CYAN, WHITE = xrange(30, 38)

        engine = efc(configuration, prefix, **kwargs)

        def color_sql(sql):

            COLOR_SEQ = "\033[1;%dm"

            COLOR_SQL = YELLOW

            normal = '\x1b[0m'

            return ''.join([COLOR_SEQ % COLOR_SQL, sql, normal])

        if configuration['debug']:

            #attach events only for debug configuration

            def before_cursor_execute(conn, cursor, statement,

                                    parameters, context, executemany):

                context._query_start_time = time.time()

                log.info(color_sql(">>>>> STARTING QUERY >>>>>"))

            def after_cursor_execute(conn, cursor, statement,

                                    parameters, context, executemany):

                total = time.time() - context._query_start_time

                log.info(color_sql("<<<<< TOTAL TIME: %f <<<<<" % total))

            event.listen(engine, "before_cursor_execute",

                         before_cursor_execute)

            event.listen(engine, "after_cursor_execute",

                         after_cursor_execute)

    return engine

def age(prevdate):

    """

    turns a datetime into an age string.

    :param prevdate: datetime object

    :rtype: unicode

    :returns: unicode words describing age

    """

    order = ['year', 'month', 'day', 'hour', 'minute', 'second']

    deltas = {}

    # Get date parts deltas

    now = datetime.datetime.now()

    for part in order:

        deltas[part] = getattr(now, part) - getattr(prevdate, part)

    # Fix negative offsets (there is 1 second between 10:59:59 and 11:00:00,

    # not 1 hour, -59 minutes and -59 seconds)

    for num, length in [(5, 60), (4, 60), (3, 24)]:  # seconds, minutes, hours

        part = order[num]

        carry_part = order[num - 1]

        if deltas[part] < 0:

            deltas[part] += length

            deltas[carry_part] -= 1

    # Same thing for days except that the increment depends on the (variable)

    # number of days in the month

    month_lengths = [31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31]

    if deltas['day'] < 0:

        if prevdate.month == 2 and (prevdate.year % 4 == 0 and

            (prevdate.year % 100 != 0 or prevdate.year % 400 == 0)):

            deltas['day'] += 29

"""

Set of generic validators

"""

import os

import re

import formencode

import logging

from pylons.i18n.translation import _

from webhelpers.pylonslib.secure_form import authentication_token

from formencode.validators import (

    UnicodeString, OneOf, Int, Number, Regex, Email, Bool, StringBoolean, Set,

    NotEmpty

)

from rhodecode.lib.utils import repo_name_slug

from rhodecode.model.db import RepoGroup, Repository, UsersGroup, User,\

    ChangesetStatus

from rhodecode.lib.exceptions import LdapImportError

from rhodecode.config.routing import ADMIN_PREFIX

# silence warnings and pylint

UnicodeString, OneOf, Int, Number, Regex, Email, Bool, StringBoolean, Set, \

    NotEmpty

log = logging.getLogger(__name__)

class UniqueList(formencode.FancyValidator):

    """

    Unique List !

    """

    messages = dict(

        empty=_('Value cannot be an empty list'),

        missing_value=_('Value cannot be an empty list'),

    )

    def _to_python(self, value, state):

        if isinstance(value, list):

            return value

        elif isinstance(value, set):

            return list(value)

        elif isinstance(value, tuple):

            return list(value)

        elif value is None:

            return []

        else:

            return [value]

    def empty_value(self, value):

        return []

class StateObj(object):

    """

    this is needed to translate the messages using _() in validators

    """

    _ = staticmethod(_)

def M(self, key, state=None, **kwargs):

    """

    returns string from self.message based on given key,

        }

        def validate_python(self, value, state):

            repo_type = value.get('repo_type')

            url = value.get('clone_uri')

            if not url:

                pass

            else:

                try:

                    url_handler(repo_type, url, make_ui('db', clear_session=False))

                except Exception:

                    log.exception('Url validation failed')

                    msg = M(self, 'clone_uri')

                    raise formencode.Invalid(msg, value, state,

                        error_dict=dict(clone_uri=msg)

                    )

    return _validator

def ValidForkType(old_data={}):

    class _validator(formencode.validators.FancyValidator):

        messages = {

            'invalid_fork_type': _(u'Fork have to be the same type as parent')

        }

        def validate_python(self, value, state):

            if old_data['repo_type'] != value:

                msg = M(self, 'invalid_fork_type', state)

                raise formencode.Invalid(msg, value, state,

                    error_dict=dict(repo_type=msg)

                )

    return _validator

def ValidPerms(type_='repo'):

    if type_ == 'group':

        EMPTY_PERM = 'group.none'

    elif type_ == 'repo':

        EMPTY_PERM = 'repository.none'

    class _validator(formencode.validators.FancyValidator):

        messages = {

            'perm_new_member_name':

                _(u'This username or users group name is not valid')

        }

        def to_python(self, value, state):

            # build a list of permission to update and new permission to create

                if k.startswith('perm_new_member'):

                    member = k[7:]

                    t = {'u': 'user',

                         'g': 'users_group'

                    }[k[0]]

                    if member == 'default':

                        if value.get('private'):

                            # set none for default when updating to

                            # private repo

                            v = EMPTY_PERM

            # update permissions

            for k, v, t in perms_new:

                try:

                    if t is 'user':

                        self.user_db = User.query()\

                            .filter(User.active == True)\

                            .filter(User.username == k).one()

                    if t is 'users_group':

                        self.user_db = UsersGroup.query()\

                            .filter(UsersGroup.users_group_active == True)\

                            .filter(UsersGroup.users_group_name == k).one()

                except Exception:

                    log.exception('Updated permission failed')

                    msg = M(self, 'perm_new_member_type', state)

                    raise formencode.Invalid(msg, value, state,

                        error_dict=dict(perm_new_member_name=msg)

                    )

            return value

    return _validator

def ValidSettings():

    class _validator(formencode.validators.FancyValidator):

        def _to_python(self, value, state):

            # settings  form can't edit user

            if 'user' in value:

                del value['user']

            return value

        def validate_python(self, value, state):

            pass

    return _validator

def ValidPath():

    class _validator(formencode.validators.FancyValidator):

        messages = {

            'invalid_path': _(u'This is not a valid path')

        }

        def validate_python(self, value, state):

            if not os.path.isdir(value):

                msg = M(self, 'invalid_path', state)

                raise formencode.Invalid(msg, value, state,

                    error_dict=dict(paths_root_path=msg)

                )

	        }

	    }

	}

};

var deleteNotification = function(url, notification_id,callbacks){

    var callback = { 

		success:function(o){

		    var obj = YUD.get(String("notification_"+notification_id));

		    if(obj.parentNode !== undefined){

				obj.parentNode.removeChild(obj);

			}

			_run_callbacks(callbacks);

		},

	    failure:function(o){

	        alert("error");

	    },

	};

    var postData = '_method=delete';

    var sUrl = url.replace('__NOTIFICATION_ID__',notification_id);

    var request = YAHOO.util.Connect.asyncRequest('POST', sUrl, 

    											  callback, postData);

};	

var readNotification = function(url, notification_id,callbacks){

    var callback = { 

		success:function(o){

		    var obj = YUD.get(String("notification_"+notification_id));

		    YUD.removeClass(obj, 'unread');

		    var r_button = YUD.getElementsByClassName('read-notification',null,obj.children[0])[0];

		    if(r_button.parentNode !== undefined){

		    	r_button.parentNode.removeChild(r_button);

			}		    

			_run_callbacks(callbacks);

		},

	    failure:function(o){

	        alert("error");

	    },

	};

    var postData = '_method=put';

    var sUrl = url.replace('__NOTIFICATION_ID__',notification_id);

    var request = YAHOO.util.Connect.asyncRequest('POST', sUrl, 

    											  callback, postData);

};	

/** MEMBERS AUTOCOMPLETE WIDGET **/

    var myUsers = users_list;

    var myGroups = groups_list;

    // Define a custom search function for the DataSource of users

    var matchUsers = function (sQuery) {

            // Case insensitive matching

            var query = sQuery.toLowerCase();

            var i = 0;

            var l = myUsers.length;

            var matches = [];

            // Match against each name of each contact

            for (; i < l; i++) {

                contact = myUsers[i];

                if (((contact.fname+"").toLowerCase().indexOf(query) > -1) || 

                   	 ((contact.lname+"").toLowerCase().indexOf(query) > -1) || 

                   	 ((contact.nname) && ((contact.nname).toLowerCase().indexOf(query) > -1))) {

                       matches[matches.length] = contact;

                   }

            }

            return matches;

        };

    // Define a custom search function for the DataSource of usersGroups

    var matchGroups = function (sQuery) {

            // Case insensitive matching

            var query = sQuery.toLowerCase();

            var i = 0;

            var l = myGroups.length;

            var matches = [];

            // Match against each name of each contact

            for (; i < l; i++) {

                matched_group = myGroups[i];

                if (matched_group.grname.toLowerCase().indexOf(query) > -1) {

                    matches[matches.length] = matched_group;

                }

            }

            return matches;

        };

    //match all

    var matchAll = function (sQuery) {

            u = matchUsers(sQuery);

            g = matchGroups(sQuery);

            return u.concat(g);

        };

    // DataScheme for members

    var memberDS = new YAHOO.util.FunctionDataSource(matchAll);

    memberDS.responseSchema = {

        fields: ["id", "fname", "lname", "nname", "grname", "grmembers", "gravatar_lnk"]

    };

    // DataScheme for owner

    var ownerDS = new YAHOO.util.FunctionDataSource(matchUsers);

    ownerDS.responseSchema = {

        fields: ["id", "fname", "lname", "nname", "gravatar_lnk"]

    };

    // Instantiate AutoComplete for perms

    membersAC.useShadow = false;

    membersAC.resultTypeList = false;

    membersAC.animVert = false;

    membersAC.animHoriz = false;    

    membersAC.animSpeed = 0.1;

    // Instantiate AutoComplete for owner

    var ownerAC = new YAHOO.widget.AutoComplete("user", "owner_container", ownerDS);

    ownerAC.useShadow = false;

    ownerAC.resultTypeList = false;

    ownerAC.animVert = false;

    ownerAC.animHoriz = false;

    ownerAC.animSpeed = 0.1;

    // Helper highlight function for the formatter

    var highlightMatch = function (full, snippet, matchindex) {

            return full.substring(0, matchindex) 

            + "<span class='match'>" 

            + full.substr(matchindex, snippet.length) 

            + "</span>" + full.substring(matchindex + snippet.length);

        };

    // Custom formatter to highlight the matching letters

    var custom_formatter = function (oResultData, sQuery, sResultMatch) {

            var query = sQuery.toLowerCase();

            var _gravatar = function(res, em, group){

            	if (group !== undefined){

            		em = '/images/icons/group.png'

            	}

            	tmpl = '<div class="ac-container-wrap"><img class="perm-gravatar-ac" src="{0}"/>{1}</div>'

            	return tmpl.format(em,res)

            }

            // group

            if (oResultData.grname != undefined) {

                var grname = oResultData.grname;

                var grmembers = oResultData.grmembers;

                var grnameMatchIndex = grname.toLowerCase().indexOf(query);

                var grprefix = "{0}: ".format(_TM['Group']);

                var grsuffix = " (" + grmembers + "  )";

                var grsuffix = " ({0}  {1})".format(grmembers, _TM['members']);

                if (grnameMatchIndex > -1) {

                    return _gravatar(grprefix + highlightMatch(grname, query, grnameMatchIndex) + grsuffix,null,true);

                }

			    return _gravatar(grprefix + oResultData.grname + grsuffix, null,true);

            // Users

            } else if (oResultData.nname != undefined) {

                var fname = oResultData.fname || "";

                var lname = oResultData.lname || "";

                var nname = oResultData.nname;

                // Guard against null value

                var fnameMatchIndex = fname.toLowerCase().indexOf(query),

                    lnameMatchIndex = lname.toLowerCase().indexOf(query),

                    nnameMatchIndex = nname.toLowerCase().indexOf(query),

                    displayfname, displaylname, displaynname;

                if (fnameMatchIndex > -1) {

                    displayfname = highlightMatch(fname, query, fnameMatchIndex);

                } else {

                    displayfname = fname;

                }

                if (lnameMatchIndex > -1) {

                    displaylname = highlightMatch(lname, query, lnameMatchIndex);

                } else {

                    displaylname = lname;

                }

                if (nnameMatchIndex > -1) {

                    displaynname = "(" + highlightMatch(nname, query, nnameMatchIndex) + ")";

                } else {

                    displaynname = nname ? "(" + nname + ")" : "";

                }

                return _gravatar(displayfname + " " + displaylname + " " + displaynname, oResultData.gravatar_lnk);

            } else {

                return '';

            }

        };

    membersAC.formatResult = custom_formatter;

    ownerAC.formatResult = custom_formatter;

    var myHandler = function (sType, aArgs) {

            var myAC = aArgs[0]; // reference back to the AC instance

            var elLI = aArgs[1]; // reference to the selected LI element

            var oData = aArgs[2]; // object literal of selected item's result data

            //fill the autocomplete with value

            if (oData.nname != undefined) {

                //users

                myAC.getInputEl().value = oData.nname;

            } else {

                //groups

                myAC.getInputEl().value = oData.grname;

            }

        };

    membersAC.itemSelectEvent.subscribe(myHandler);

    if(ownerAC.itemSelectEvent){

    	ownerAC.itemSelectEvent.subscribe(myHandler);

    }

    return {

        memberDS: memberDS,

        ownerDS: ownerDS,

        membersAC: membersAC,

        ownerAC: ownerAC,

    };

}

var MentionsAutoComplete = function (divid, cont, users_list, groups_list) {

    var myUsers = users_list;

    var myGroups = groups_list;

    // Define a custom search function for the DataSource of users

    var matchUsers = function (sQuery) {

    	    var org_sQuery = sQuery;

    	    if(this.mentionQuery == null){

    	    	return []    	    	

    	    }

    	    sQuery = this.mentionQuery;

            // Case insensitive matching

            var query = sQuery.toLowerCase();

            var i = 0;

            var l = myUsers.length;

            var matches = [];

            // Match against each name of each contact

            for (; i < l; i++) {

                contact = myUsers[i];

                if (((contact.fname+"").toLowerCase().indexOf(query) > -1) || 

                	 ((contact.lname+"").toLowerCase().indexOf(query) > -1) || 

                	 ((contact.nname) && ((contact.nname).toLowerCase().indexOf(query) > -1))) {

                    matches[matches.length] = contact;

                }

            }

            return matches

        };

    //match all

    var matchAll = function (sQuery) {

    // extract name from table

    a_ = a_.children[0].innerHTML;

    b_ = b_.children[0].innerHTML;      

    var comp = YAHOO.util.Sort.compare;

    var compState = comp(a_, b_, desc);

    return compState;

};

var groupNameSort = function(a, b, desc, field) {

    var a_ = fromHTML(a.getData(field));

    var b_ = fromHTML(b.getData(field));

    // extract name from table

    a_ = get_group_name(a_)

    b_ = get_group_name(b_)          

    var comp = YAHOO.util.Sort.compare;

    var compState = comp(a_, b_, desc);

    return compState;

};

var dateSort = function(a, b, desc, field) {

    var a_ = fromHTML(a.getData(field));

    var b_ = fromHTML(b.getData(field));

    // extract name from table

    a_ = get_date(a_)

    b_ = get_date(b_)          

    var comp = YAHOO.util.Sort.compare;

    var compState = comp(a_, b_, desc);

    return compState;

};

var linkSort = function(a, b, desc, field) {

	  var a_ = fromHTML(a.getData(field));

	  var b_ = fromHTML(a.getData(field));

	  // extract url text from string nodes 

	  a_ = get_link(a_)

	  b_ = get_link(b_)

	  var comp = YAHOO.util.Sort.compare;

	  var compState = comp(a_, b_, desc);

	  return compState;

}

/* Multi selectors */

var MultiSelectWidget = function(selected_id, available_id, form_id){

	//definition of containers ID's

	var selected_container = selected_id;

	var available_container = available_id;

	//temp container for selected storage.

	var cache = new Array();

	var av_cache = new Array();

	var c =  YUD.get(selected_container);

	var ac = YUD.get(available_container);

	//get only selected options for further fullfilment

	for(var i = 0;node =c.options[i];i++){

	    if(node.selected){

	        //push selected to my temp storage left overs :)

	        cache.push(node);

	    }

	}

	//get all available options to cache

	for(var i = 0;node =ac.options[i];i++){

	        //push selected to my temp storage left overs :)

	        av_cache.push(node);

	}

	//fill available only with those not in choosen

	ac.options.length=0;

	tmp_cache = new Array();

	for(var i = 0;node = av_cache[i];i++){

	    var add = true;

	    for(var i2 = 0;node_2 = cache[i2];i2++){

	        if(node.value == node_2.value){

	            add=false;

	            break;

	        }

	    }

	    if(add){

	        tmp_cache.push(new Option(node.text, node.value, false, false));

	    }

	}

	for(var i = 0;node = tmp_cache[i];i++){

                    <span class="private_repo_msg">

                    ${_('private repository')}

                    </span>

                </td>

                <td class="private_repo_msg"><img style="vertical-align:bottom" src="${h.url('/images/icons/user.png')}"/>${_('default')}</td>

            </tr>

        %else:

        <tr id="id${id(r2p.user.username)}">

            <td>${h.radio('u_perm_%s' % r2p.user.username,'repository.none')}</td>

            <td>${h.radio('u_perm_%s' % r2p.user.username,'repository.read')}</td>

            <td>${h.radio('u_perm_%s' % r2p.user.username,'repository.write')}</td>

            <td>${h.radio('u_perm_%s' % r2p.user.username,'repository.admin')}</td>

            <td style="white-space: nowrap;">

                <img class="perm-gravatar" src="${h.gravatar_url(r2p.user.email,14)}"/>${r2p.user.username if r2p.user.username != 'default' else _('default')}

            </td>

            <td>

              %if r2p.user.username !='default':

                <span class="delete_icon action_button" onclick="ajaxActionUser(${r2p.user.user_id},'${'id%s'%id(r2p.user.username)}')">

                ${_('revoke')}

                </span>

              %endif

            </td>

        </tr>

        %endif

    %endfor

    ## USERS GROUPS

    %for g2p in c.repo_info.users_group_to_perm:

        <tr id="id${id(g2p.users_group.users_group_name)}">

            <td>${h.radio('g_perm_%s' % g2p.users_group.users_group_name,'repository.none')}</td>

            <td>${h.radio('g_perm_%s' % g2p.users_group.users_group_name,'repository.read')}</td>

            <td>${h.radio('g_perm_%s' % g2p.users_group.users_group_name,'repository.write')}</td>

            <td>${h.radio('g_perm_%s' % g2p.users_group.users_group_name,'repository.admin')}</td>

            <td style="white-space: nowrap;">

                <img class="perm-gravatar" src="${h.url('/images/icons/group.png')}"/>

                %if h.HasPermissionAny('hg.admin')():

                 <a href="${h.url('edit_users_group',id=g2p.users_group.users_group_id)}">${g2p.users_group.users_group_name}</a>

                %else:

                 ${g2p.users_group.users_group_name}

                %endif

            </td>

            <td>

                <span class="delete_icon action_button" onclick="ajaxActionUsersGroup(${g2p.users_group.users_group_id},'${'id%s'%id(g2p.users_group.users_group_name)}')">

                ${_('revoke')}

                </span>

            </td>

        </tr>

    %endfor