.. _changelog:
=========
Changelog
1.4.4 (**2012-XX-XX**)
----------------------
:status: in-progress
:branch: beta
news
++++
- obfuscate db password in logs for engine connection string
- #574 Show pull request status also in shortlog (if any)
- remember selected tab in my account page
- Bumped mercurial version to 2.3.2
fixes
+++++
- Add git version detection to warn users that Git used in system is to
old. Ref #588 - also show git version in system details in settings page
- fixed files quick filter links
- #590 Add GET flag that controls the way the diff are generated, for pull
requests we want to use non-bundle based diffs, That are far better for
doing code reviews. The /compare url still uses bundle compare for full
comparison including the incoming changesets
- Fixed #585, checks for status of revision where to strict, and made
opening pull request with those revision impossible due to previously set
status. Checks now are made also for the repository.
- fixes #591 git backend was causing encoding errors when handling binary
files - added a test case for VCS lib tests
- fixed #597 commits in future get negative age.
1.4.3 (**2012-09-28**)
- #558 Added config file to hooks extra data
- bumped mercurial version to 2.3.1
- #518 added possibility of specifying multiple patterns for issues
- update codemirror to latest version
- fixed #570 explicit users group permissions can overwrite owner permissions
- fixed #578 set proper PATH with current Python for Git
hooks to execute within same Python as RhodeCode
- fixed issue with Git bare repos that ends with .git in name
1.4.2 (**2012-09-12**)
- added option to menu to quick lock/unlock repository for users that have
write access to
- Implemented permissions for writing to repo
groups. Now only write access to group allows to create a repostiory
within that group
- #565 Add support for {netloc} and {scheme} to alternative_gravatar_url
- updated translation for zh_CN
- fixed visual permissions check on repos groups inside groups
- fixed issues with non-ascii search terms in search, and indexers
- fixed parsing of page number in GET parameters
- fixed issues with generating pull-request overview for repos with
bookmarks and tags, also preview doesn't loose chosen revision from
select dropdown
1.4.1 (**2012-09-07**)
- always put a comment about code-review status change even if user send
empty data
- modified_on column saves repository update and it's going to be used
later for light version of main page ref #500
- pull request notifications send much nicer emails with details about pull
request
- #551 show breadcrumbs in summary view for repositories inside a group
- fixed migrations of permissions that can lead to inconsistency.
Some users sent feedback that after upgrading from older versions issues
with updating default permissions occurred. RhodeCode detects that now and
resets default user permission to initial state if there is a need for that.
Also forces users to set the default value for new forking permission.
- #535 improved apache wsgi example configuration in docs
- fixes #550 mercurial repositories comparision failed when origin repo had
additional not-common changesets
- fixed status of code-review in preview windows of pull request
- git forks were not initialized at bare repos
- fixes #555 fixes issues with comparing non-related repositories
- fixes #557 follower counter always counts up
- fixed issue #560 require push ssl checkbox wasn't shown when option was
enabled
- fixed #559
- fixed issue #559 fixed bug in routing that mapped repo names with <name>_<num> in name as
if it was a request to url by repository ID
1.4.0 (**2012-09-03**)
- new codereview system
- email map, allowing users to have multiple email addresses mapped into
their accounts
- improved git-hook system. Now all actions for git are logged into journal
including pushed revisions, user and IP address
- changed setup-app into setup-rhodecode and added default options to it.
- new git repos are created as bare now by default
- #464 added links to groups in permission box
- #465 mentions autocomplete inside comments boxes
- #469 added --update-only option to whoosh to re-index only given list
of repos in index
- rhodecode-api CLI client
- new git http protocol replaced buggy dulwich implementation.
Now based on pygrack & gitweb
- Improved RSS/ATOM feeds. Discoverable by browsers using proper headers, and
reformated based on user suggestions. Additional rss/atom feeds for user
journal
- various i18n improvements
- #478 permissions overview for admin in user edit view
- File view now displays small gravatars off all authors of given file
- Implemented landing revisions. Each repository will get landing_rev attribute
that defines 'default' revision/branch for generating readme files
- Implemented #509, RhodeCode enforces SSL for push/pulling if requested at
earliest possible call.
- Import remote svn repositories to mercurial using hgsubversion.
- Fixed #508 RhodeCode now has a option to explicitly set forking permissions
- RhodeCode can use alternative server for generating avatar icons
- implemented repositories locking. Pull locks, push unlocks. Also can be done
via API calls
- #538 form for permissions can handle multiple users at once
- improved translations
- fixes issue #455 Creating an archive generates an exception on Windows
- fixes #448 Download ZIP archive keeps file in /tmp open and results
in out of disk space
- fixes issue #454 Search results under Windows include proceeding
backslash
- fixed issue #450. Rhodecode no longer will crash when bad revision is
present in journal data.
- fix for issue #417, git execution was broken on windows for certain
commands.
- fixed #413. Don't disable .git directory for bare repos on deleting
- fixed issue #459. Changed the way of obtaining logger in reindex task.
- fixed #453 added ID field in whoosh SCHEMA that solves the issue of
reindexing modified files
- fixed #481 rhodecode emails are sent without Date header
- fixed #458 wrong count when no repos are present
- fixed issue #492 missing `\ No newline at end of file` test at the end of
new chunk in html diff
- full text search now works also for commit messages
1.3.6 (**2012-05-17**)
- chinese traditional translation
- changed setup-app into setup-rhodecode and added arguments for auto-setup
mode that doesn't need user interaction
- fixed no scm found warning
- fixed __future__ import error on rcextensions
- made simplejson required lib for speedup on JSON encoding
- fixes #449 bad regex could get more than revisions from parsing history
- don't clear DB session when CELERY_EAGER is turned ON
1.3.5 (**2012-05-10**)
- use ext_json for json module
- unified annotation view with file source view
- notification improvements, better inbox + css
- #419 don't strip passwords for login forms, make rhodecode
more compatible with LDAP servers
- Added HTTP_X_FORWARDED_FOR as another method of extracting
IP for pull/push logs. - moved all to base controller
- #415: Adding comment to changeset causes reload.
Comments are now added via ajax and doesn't reload the page
- #374 LDAP config is discarded when LDAP can't be activated
- limited push/pull operations are now logged for git in the journal
- bumped mercurial to 2.2.X series
- added support for displaying submodules in file-browser
- #421 added bookmarks in changelog view
- fixed dev-version marker for stable when served from source codes
- fixed missing permission checks on show forks page
- #418 cast to unicode fixes in notification objects
- #426 fixed mention extracting regex
- fixed remote-pulling for git remotes remopositories
- fixed #434: Error when accessing files or changesets of a git repository
with submodules
- fixed issue with empty APIKEYS for users after registration ref. #438
- fixed issue with getting README files from git repositories
@@ -125,383 +125,394 @@ def detect_mode(line, default):
elif line.endswith('\r'):
return 1
else:
return default
def generate_api_key(username, salt=None):
"""
Generates unique API key for given username, if salt is not given
it'll be generated from some random string
:param username: username as string
:param salt: salt to hash generate KEY
:rtype: str
:returns: sha1 hash from username+salt
from tempfile import _RandomNameSequence
import hashlib
if salt is None:
salt = _RandomNameSequence().next()
return hashlib.sha1(username + salt).hexdigest()
def safe_int(val, default=None):
Returns int() of val if val is not convertable to int use default
instead
:param val:
:param default:
try:
val = int(val)
except ValueError:
val = default
return val
def safe_unicode(str_, from_encoding=None):
safe unicode function. Does few trick to turn str_ into unicode
In case of UnicodeDecode error we try to return it with encoding detected
by chardet library if it fails fallback to unicode with errors replaced
:param str_: string to decode
:rtype: unicode
:returns: unicode object
if isinstance(str_, unicode):
return str_
if not from_encoding:
import rhodecode
DEFAULT_ENCODING = rhodecode.CONFIG.get('default_encoding','utf8')
from_encoding = DEFAULT_ENCODING
return unicode(str_)
except UnicodeDecodeError:
pass
return unicode(str_, from_encoding)
import chardet
encoding = chardet.detect(str_)['encoding']
if encoding is None:
raise Exception()
return str_.decode(encoding)
except (ImportError, UnicodeDecodeError, Exception):
return unicode(str_, from_encoding, 'replace')
def safe_str(unicode_, to_encoding=None):
safe str function. Does few trick to turn unicode_ into string
In case of UnicodeEncodeError we try to return it with encoding detected
by chardet library if it fails fallback to string with errors replaced
:param unicode_: unicode to encode
:returns: str object
# if it's not basestr cast to str
if not isinstance(unicode_, basestring):
return str(unicode_)
if isinstance(unicode_, str):
return unicode_
if not to_encoding:
to_encoding = DEFAULT_ENCODING
return unicode_.encode(to_encoding)
except UnicodeEncodeError:
encoding = chardet.detect(unicode_)['encoding']
raise UnicodeEncodeError()
return unicode_.encode(encoding)
except (ImportError, UnicodeEncodeError):
return unicode_.encode(to_encoding, 'replace')
return safe_str
def engine_from_config(configuration, prefix='sqlalchemy.', **kwargs):
Custom engine_from_config functions that makes sure we use NullPool for
file based sqlite databases. This prevents errors on sqlite. This only
applies to sqlalchemy versions < 0.7.0
import sqlalchemy
from sqlalchemy import engine_from_config as efc
import logging
if int(sqlalchemy.__version__.split('.')[1]) < 7:
# This solution should work for sqlalchemy < 0.7.0, and should use
# proxy=TimerProxy() for execution time profiling
from sqlalchemy.pool import NullPool
url = configuration[prefix + 'url']
if url.startswith('sqlite'):
kwargs.update({'poolclass': NullPool})
return efc(configuration, prefix, **kwargs)
import time
from sqlalchemy import event
from sqlalchemy.engine import Engine
log = logging.getLogger('sqlalchemy.engine')
BLACK, RED, GREEN, YELLOW, BLUE, MAGENTA, CYAN, WHITE = xrange(30, 38)
engine = efc(configuration, prefix, **kwargs)
def color_sql(sql):
COLOR_SEQ = "\033[1;%dm"
COLOR_SQL = YELLOW
normal = '\x1b[0m'
return ''.join([COLOR_SEQ % COLOR_SQL, sql, normal])
if configuration['debug']:
#attach events only for debug configuration
def before_cursor_execute(conn, cursor, statement,
parameters, context, executemany):
context._query_start_time = time.time()
log.info(color_sql(">>>>> STARTING QUERY >>>>>"))
def after_cursor_execute(conn, cursor, statement,
total = time.time() - context._query_start_time
log.info(color_sql("<<<<< TOTAL TIME: %f <<<<<" % total))
event.listen(engine, "before_cursor_execute",
before_cursor_execute)
event.listen(engine, "after_cursor_execute",
after_cursor_execute)
return engine
def age(prevdate):
turns a datetime into an age string.
:param prevdate: datetime object
:returns: unicode words describing age
order = ['year', 'month', 'day', 'hour', 'minute', 'second']
deltas = {}
future = False
# Get date parts deltas
now = datetime.datetime.now()
if prevdate > now:
now, prevdate = prevdate, now
future = True
for part in order:
deltas[part] = getattr(now, part) - getattr(prevdate, part)
# Fix negative offsets (there is 1 second between 10:59:59 and 11:00:00,
# not 1 hour, -59 minutes and -59 seconds)
for num, length in [(5, 60), (4, 60), (3, 24)]: # seconds, minutes, hours
part = order[num]
carry_part = order[num - 1]
if deltas[part] < 0:
deltas[part] += length
deltas[carry_part] -= 1
# Same thing for days except that the increment depends on the (variable)
# number of days in the month
month_lengths = [31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31]
if deltas['day'] < 0:
if prevdate.month == 2 and (prevdate.year % 4 == 0 and
(prevdate.year % 100 != 0 or prevdate.year % 400 == 0)):
deltas['day'] += 29
deltas['day'] += month_lengths[prevdate.month - 1]
deltas['month'] -= 1
if deltas['month'] < 0:
deltas['month'] += 12
deltas['year'] -= 1
# Format the result
fmt_funcs = {
'year': lambda d: ungettext(u'%d year', '%d years', d) % d,
'month': lambda d: ungettext(u'%d month', '%d months', d) % d,
'day': lambda d: ungettext(u'%d day', '%d days', d) % d,
'hour': lambda d: ungettext(u'%d hour', '%d hours', d) % d,
'minute': lambda d: ungettext(u'%d minute', '%d minutes', d) % d,
'second': lambda d: ungettext(u'%d second', '%d seconds', d) % d,
}
for i, part in enumerate(order):
value = deltas[part]
if value == 0:
continue
if i < 5:
sub_part = order[i + 1]
sub_value = deltas[sub_part]
sub_value = 0
if sub_value == 0:
return _(u'%s ago') % fmt_funcs[part](value)
return _(u'%s and %s ago') % (fmt_funcs[part](value),
fmt_funcs[sub_part](sub_value))
if future:
return _(u'in %s') % fmt_funcs[part](value)
return _(u'in %s and %s') % (fmt_funcs[part](value),
return _(u'just now')
def uri_filter(uri):
Removes user:password from given url string
:param uri:
:returns: filtered list of strings
if not uri:
return ''
proto = ''
for pat in ('https://', 'http://'):
if uri.startswith(pat):
uri = uri[len(pat):]
proto = pat
break
# remove passwords and username
uri = uri[uri.find('@') + 1:]
# get the port
cred_pos = uri.find(':')
if cred_pos == -1:
host, port = uri, None
host, port = uri[:cred_pos], uri[cred_pos + 1:]
return filter(None, [proto, host, port])
def credentials_filter(uri):
Returns a url with removed credentials
uri = uri_filter(uri)
#check if we have port
if len(uri) > 2 and uri[2]:
uri[2] = ':' + uri[2]
return ''.join(uri)
def get_changeset_safe(repo, rev):
Safe version of get_changeset if this changeset doesn't exists for a
repo it returns a Dummy one instead
:param repo:
:param rev:
from rhodecode.lib.vcs.backends.base import BaseRepository
from rhodecode.lib.vcs.exceptions import RepositoryError
from rhodecode.lib.vcs.backends.base import EmptyChangeset
if not isinstance(repo, BaseRepository):
raise Exception('You must pass an Repository '
'object as first argument got %s', type(repo))
cs = repo.get_changeset(rev)
except RepositoryError:
cs = EmptyChangeset(requested_revision=rev)
return cs
def datetime_to_time(dt):
if dt:
return time.mktime(dt.timetuple())
def time_to_datetime(tm):
if tm:
if isinstance(tm, basestring):
tm = float(tm)
return
return datetime.datetime.fromtimestamp(tm)
MENTIONS_REGEX = r'(?:^@|\s@)([a-zA-Z0-9]{1}[a-zA-Z0-9\-\_\.]+)(?:\s{1})'
def extract_mentioned_users(s):
Returns unique usernames from given string s that have @mention
:param s: string to get mentions
usrs = set()
for username in re.findall(MENTIONS_REGEX, s):
usrs.add(username)
return sorted(list(usrs), key=lambda k: k.lower())
class AttributeDict(dict):
def __getattr__(self, attr):
return self.get(attr, None)
__setattr__ = dict.__setitem__
__delattr__ = dict.__delitem__
def fix_PATH(os_=None):
Get current active python path, and append it to PATH variable to fix issues
of subprocess calls and different python versions
import sys
if os_ is None:
import os
os = os_
cur_path = os.path.split(sys.executable)[0]
if not os.environ['PATH'].startswith(cur_path):
os.environ['PATH'] = '%s:%s' % (cur_path, os.environ['PATH'])
def obfuscate_url_pw(engine):
from sqlalchemy.engine import url
url = url.make_url(engine)
if url.password:
url.password = 'XXXXX'
return str(url)
\ No newline at end of file
# -*- coding: utf-8 -*-
rhodecode.tests.test_libs
~~~~~~~~~~~~~~~~~~~~~~~~~
Package for testing various lib/helper functions in rhodecode
:created_on: Jun 9, 2011
:copyright: (C) 2011-2012 Marcin Kuzminski <marcin@python-works.com>
:license: GPLv3, see COPYING for more details.
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from __future__ import with_statement
import unittest
import datetime
import mock
from rhodecode.tests import *
proto = 'http'
TEST_URLS = [
('%s://127.0.0.1' % proto, ['%s://' % proto, '127.0.0.1'],
'%s://127.0.0.1' % proto),
('%s://marcink@127.0.0.1' % proto, ['%s://' % proto, '127.0.0.1'],
('%s://marcink:pass@127.0.0.1' % proto, ['%s://' % proto, '127.0.0.1'],
('%s://127.0.0.1:8080' % proto, ['%s://' % proto, '127.0.0.1', '8080'],
'%s://127.0.0.1:8080' % proto),
('%s://domain.org' % proto, ['%s://' % proto, 'domain.org'],
'%s://domain.org' % proto),
('%s://user:pass@domain.org:8080' % proto, ['%s://' % proto, 'domain.org',
'8080'],
'%s://domain.org:8080' % proto),
]
proto = 'https'
TEST_URLS += [
class TestLibs(unittest.TestCase):
def test_uri_filter(self):
from rhodecode.lib.utils2 import uri_filter
for url in TEST_URLS:
self.assertEqual(uri_filter(url[0]), url[1])
def test_credentials_filter(self):
from rhodecode.lib.utils2 import credentials_filter
self.assertEqual(credentials_filter(url[0]), url[2])
def test_str2bool(self):
from rhodecode.lib.utils2 import str2bool
test_cases = [
('t', True),
('true', True),
('y', True),
('yes', True),
('on', True),
('1', True),
('Y', True),
('yeS', True),
('TRUE', True),
('T', True),
('False', False),
('F', False),
('FALSE', False),
('0', False),
('-1', False),
('', False), ]
for case in test_cases:
self.assertEqual(str2bool(case[0]), case[1])
def test_mention_extractor(self):
from rhodecode.lib.utils2 import extract_mentioned_users
sample = (
"@first hi there @marcink here's my email marcin@email.com "
"@lukaszb check @one_more22 it pls @ ttwelve @D[] @one@two@three "
"@MARCIN @maRCiN @2one_more22 @john please see this http://org.pl "
"@marian.user just do it @marco-polo and next extract @marco_polo "
"user.dot hej ! not-needed maril@domain.org"
)
s = sorted([
'first', 'marcink', 'lukaszb', 'one_more22', 'MARCIN', 'maRCiN', 'john',
'marian.user', 'marco-polo', 'marco_polo'
], key=lambda k: k.lower())
self.assertEqual(s, extract_mentioned_users(sample))
def test_age(self):
import calendar
from rhodecode.lib.utils2 import age
n = datetime.datetime.now()
delt = lambda *args, **kwargs: datetime.timedelta(*args, **kwargs)
self.assertEqual(age(n), u'just now')
self.assertEqual(age(n - delt(seconds=1)), u'1 second ago')
self.assertEqual(age(n - delt(seconds=60 * 2)), u'2 minutes ago')
self.assertEqual(age(n - delt(hours=1)), u'1 hour ago')
self.assertEqual(age(n - delt(hours=24)), u'1 day ago')
self.assertEqual(age(n - delt(hours=24 * 5)), u'5 days ago')
self.assertEqual(age(n - delt(hours=24 * (calendar.mdays[n.month-1] + 2))),
self.assertEqual(age(n - delt(hours=24 * (calendar.mdays[n.month - 1] + 2))),
u'1 month and 2 days ago')
self.assertEqual(age(n - delt(hours=24 * 400)), u'1 year and 1 month ago')
def test_age_in_future(self):
self.assertEqual(age(n + delt(seconds=1)), u'in 1 second')
self.assertEqual(age(n + delt(seconds=60 * 2)), u'in 2 minutes')
self.assertEqual(age(n + delt(hours=1)), u'in 1 hour')
self.assertEqual(age(n + delt(hours=24)), u'in 1 day')
self.assertEqual(age(n + delt(hours=24 * 5)), u'in 5 days')
self.assertEqual(age(n + delt(hours=24 * (calendar.mdays[n.month - 1] + 2))),
u'in 1 month and 1 days')
self.assertEqual(age(n + delt(hours=24 * 400)), u'in 1 year and 1 month')
def test_tag_exctrator(self):
"hello pta[tag] gog [[]] [[] sda ero[or]d [me =>>< sa]"
"[requires] [stale] [see<>=>] [see => http://url.com]"
"[requires => url] [lang => python] [just a tag]"
"[,d] [ => ULR ] [obsolete] [desc]]"
from rhodecode.lib.helpers import desc_stylize
res = desc_stylize(sample)
self.assertTrue('<div class="metatag" tag="tag">tag</div>' in res)
self.assertTrue('<div class="metatag" tag="obsolete">obsolete</div>' in res)
self.assertTrue('<div class="metatag" tag="stale">stale</div>' in res)
self.assertTrue('<div class="metatag" tag="lang">python</div>' in res)
self.assertTrue('<div class="metatag" tag="requires">requires => <a href="/url">url</a></div>' in res)
def test_alternative_gravatar(self):
from rhodecode.lib.helpers import gravatar_url
_md5 = lambda s: hashlib.md5(s).hexdigest()
def fake_conf(**kwargs):
from pylons import config
config['app_conf'] = {}
config['app_conf']['use_gravatar'] = True
config['app_conf'].update(kwargs)
return config
class fake_url():
@classmethod
def current(cls, *args, **kwargs):
return 'https://server.com'
with mock.patch('pylons.url', fake_url):
fake = fake_conf(alternative_gravatar_url='http://test.com/{email}')
with mock.patch('pylons.config', fake):
from pylons import url
assert url.current() == 'https://server.com'
grav = gravatar_url(email_address='test@foo.com', size=24)
assert grav == 'http://test.com/test@foo.com'
fake = fake_conf(alternative_gravatar_url='http://test.com/{md5email}')
em = 'test@foo.com'
grav = gravatar_url(email_address=em, size=24)
assert grav == 'http://test.com/%s' % (_md5(em))
fake = fake_conf(alternative_gravatar_url='http://test.com/{md5email}/{size}')
assert grav == 'http://test.com/%s/%s' % (_md5(em), 24)
fake = fake_conf(alternative_gravatar_url='{scheme}://{netloc}/{md5email}/{size}')
assert grav == 'https://server.com/%s/%s' % (_md5(em), 24)
Status change: