Changeset - dc2584ba5fbc
[Not reviewed]
Merge default
0 80 6
Marcin Kuzminski - 14 years ago 2012-03-28 19:54:16
marcin@python-works.com
merged beta into default branch
50 files changed:
Changeset was too big and was cut off... Show full diff anyway
0 comments (0 inline, 0 general)
.hgignore
Show inline comments
 
syntax: glob
 
*.pyc
 
*.swp
 
*.sqlite
 
*.egg-info
 
*.egg
 

	
 
syntax: regexp
 
^rcextensions
 
^build
 
^docs/build/
 
^docs/_build/
 
^data$
 
^\.settings$
 
^\.project$
 
^\.pydevproject$
 
^\.coverage$
 
^rhodecode\.db$
 
^test\.db$
 
^RhodeCode\.egg-info$
 
^rc\.ini$
 
^fabfile.py
README.rst
Show inline comments
 
=========
 
RhodeCode
 
=========
 

	
 
About
 
-----
 

	
 
``RhodeCode`` is a fast and powerful management tool for Mercurial_ and GIT_ 
 
with a built in push/pull server and full text search and code-review.
 
It works on http/https and has a built in permission/authentication system with 
 
the ability to authenticate via LDAP or ActiveDirectory. RhodeCode also provides
 
simple API so it's easy integrable with existing external systems.
 

	
 
RhodeCode is similar in some respects to github_ or bitbucket_, 
 
however RhodeCode can be run as standalone hosted application on your own server.
 
It is open source and donation ware and focuses more on providing a customized, 
 
self administered interface for Mercurial_ and GIT_  repositories. 
 
RhodeCode is powered by a vcs_ library that Lukasz Balcerzak and I created to 
 
handle multiple different version control systems.
 
RhodeCode works on *nix systems and Windows it is powered by a vcs_ library 
 
that Lukasz Balcerzak and Marcin Kuzminski created to handle multiple 
 
different version control systems.
 

	
 
RhodeCode uses `Semantic Versioning <http://semver.org/>`_
 
RhodeCode uses `PEP386 versioning http://www.python.org/dev/peps/pep-0386/`_
 

	
 
Installation
 
------------
 
Stable releases of RhodeCode are best installed via::
 

	
 
    easy_install rhodecode
 

	
 
Or::
 

	
 
    pip install rhodecode 
 

	
 
Detailed instructions and links may be found on the Installation page.
 

	
 
Please visit http://packages.python.org/RhodeCode/installation.html for
 
more details
 

	
 
RhodeCode demo
 
--------------
 

	
 
http://demo.rhodecode.org
 

	
 
The default access is anonymous but you can login to an administrative account
 
using the following credentials:
 

	
 
@@ -78,49 +79,49 @@ RhodeCode Features
 
- Users can fork other users repo. RhodeCode have also compare view to see
 
  combined changeset for all changeset made within single push.
 
- Build in commit-api let's you add, edit and commit files right from RhodeCode
 
  interface using simple editor or upload form for binaries.
 
- Mako templates let's you customize the look and feel of the application.
 
- Beautiful diffs, annotations and source code browsing all colored by pygments. 
 
  Raw diffs are made in git-diff format, including git_ binary-patches
 
- Mercurial_ branch graph and yui-flot powered graphs with zooming and statistics
 
- Admin interface with user/permission management. Admin activity journal, logs
 
  pulls, pushes, forks, registrations and other actions made by all users.
 
- Server side forks. It is possible to fork a project and modify it freely 
 
  without breaking the main repository. You can even write Your own hooks 
 
  and install them
 
- code review with notification system, inline commenting, all parsed using
 
  rst syntax
 
- rst and markdown README support for repositories  
 
- Full text search powered by Whoosh on the source files, and file names.
 
  Build in indexing daemons, with optional incremental index build
 
  (no external search servers required all in one application)
 
- Setup project descriptions and info inside built in db for easy, non 
 
  file-system operations
 
- Intelligent cache with invalidation after push or project change, provides 
 
  high performance and always up to date data.
 
- Rss / atom feeds, gravatar support, download sources as zip/tar/gz
 
- Async tasks for speed and performance using celery_ (works without them too)  
 
- Optional async tasks for speed and performance using celery_  
 
- Backup scripts can do backup of whole app and send it over scp to desired 
 
  location 
 
- Based on pylons / sqlalchemy / sqlite / whoosh / vcs
 

	
 
    
 
Incoming / Plans
 
----------------
 

	
 
- Finer granular permissions per branch, repo group or subrepo
 
- pull requests and web based merges
 
- per line file history
 
- SSH based authentication with server side key management
 
- Commit based built in wiki system
 
- More statistics and graph (global annotation + some more statistics)
 
- Other advancements as development continues (or you can of course make 
 
  additions and or requests)
 

	
 
License
 
-------
 

	
 
``RhodeCode`` is released under the GPLv3 license.
 

	
 

	
 
Getting help
development.ini
Show inline comments
 
@@ -72,48 +72,53 @@ default_encoding = utf8
 
## comment out issue_pat, issue_server, issue_prefix to enable
 

	
 
## pattern to get the issues from commit messages
 
## default one used here is #<numbers> with a regex passive group for `#`
 
## {id} will be all groups matched from this pattern
 

	
 
issue_pat = (?:\s*#)(\d+)
 

	
 
## server url to the issue, each {id} will be replaced with match
 
## fetched from the regex and {repo} is replaced with repository name
 

	
 
issue_server_link = https://myissueserver.com/{repo}/issue/{id}
 

	
 
## prefix to add to link to indicate it's an url
 
## #314 will be replaced by <issue_prefix><id>
 

	
 
issue_prefix = #
 

	
 
## instance-id prefix
 
## a prefix key for this instance used for cache invalidation when running 
 
## multiple instances of rhodecode, make sure it's globally unique for 
 
## all running rhodecode instances. Leave empty if you don't use it
 
instance_id = 
 

	
 
## alternative return HTTP header for failed authentication. Default HTTP
 
## response is 401 HTTPUnauthorized. Currently HG clients have troubles with 
 
## handling that. Set this variable to 403 to return HTTPForbidden
 
auth_ret_code =
 

	
 
####################################
 
###        CELERY CONFIG        ####
 
####################################
 
use_celery = false
 
broker.host = localhost
 
broker.vhost = rabbitmqhost
 
broker.port = 5672
 
broker.user = rabbitmq
 
broker.password = qweqwe
 

	
 
celery.imports = rhodecode.lib.celerylib.tasks
 

	
 
celery.result.backend = amqp
 
celery.result.dburi = amqp://
 
celery.result.serialier = json
 

	
 
#celery.send.task.error.emails = true
 
#celery.amqp.task.result.expires = 18000
 

	
 
celeryd.concurrency = 2
 
#celeryd.log.file = celeryd.log
 
celeryd.log.level = debug
 
celeryd.max.tasks.per.child = 1
 

	
 
@@ -150,136 +155,143 @@ beaker.cache.sql_cache_med.key_length = 
 

	
 
beaker.cache.sql_cache_long.type=file
 
beaker.cache.sql_cache_long.expire=3600
 
beaker.cache.sql_cache_long.key_length = 256
 

	
 
####################################
 
###       BEAKER SESSION        ####
 
####################################
 
## Type of storage used for the session, current types are 
 
## dbm, file, memcached, database, and memory. 
 
## The storage uses the Container API 
 
## that is also used by the cache system.
 

	
 
## db session example
 

	
 
#beaker.session.type = ext:database
 
#beaker.session.sa.url = postgresql://postgres:qwe@localhost/rhodecode
 
#beaker.session.table_name = db_session 
 

	
 
## encrypted cookie session, good for many instances
 
#beaker.session.type = cookie
 

	
 
beaker.session.type = file
 
beaker.session.key = rhodecode
 
# secure cookie requires AES python libraries
 
#beaker.session.encrypt_key = g654dcno0-9873jhgfreyu
 
#beaker.session.validate_key = 9712sds2212c--zxc123
 
beaker.session.timeout = 36000
 
beaker.session.httponly = true
 

	
 
## uncomment for https secure cookie
 
beaker.session.secure = false
 

	
 
##auto save the session to not to use .save()
 
beaker.session.auto = False
 

	
 
##true exire at browser close
 
#beaker.session.cookie_expires = 3600
 

	
 

	
 
################################################################################
 
## WARNING: *THE LINE BELOW MUST BE UNCOMMENTED ON A PRODUCTION ENVIRONMENT*  ##
 
## Debug mode will enable the interactive debugging tool, allowing ANYONE to  ##
 
## execute malicious code after an exception is raised.                       ##
 
################################################################################
 
#set debug = false
 

	
 
##################################
 
###       LOGVIEW CONFIG       ###
 
##################################
 
logview.sqlalchemy = #faa
 
logview.pylons.templating = #bfb
 
logview.pylons.util = #eee
 

	
 
#########################################################
 
### DB CONFIGS - EACH DB WILL HAVE IT'S OWN CONFIG    ###
 
#########################################################
 
#sqlalchemy.db1.url = sqlite:///%(here)s/rhodecode.db
 
sqlalchemy.db1.url = postgresql://postgres:qwe@localhost/rhodecode
 
sqlalchemy.db1.echo = false
 
sqlalchemy.db1.pool_recycle = 3600
 
sqlalchemy.convert_unicode = true
 
sqlalchemy.db1.convert_unicode = true
 

	
 
################################
 
### LOGGING CONFIGURATION   ####
 
################################
 
[loggers]
 
keys = root, routes, rhodecode, sqlalchemy, beaker, templates
 
keys = root, routes, rhodecode, sqlalchemy, beaker, templates, whoosh_indexer
 

	
 
[handlers]
 
keys = console, console_sql
 

	
 
[formatters]
 
keys = generic, color_formatter, color_formatter_sql
 

	
 
#############
 
## LOGGERS ##
 
#############
 
[logger_root]
 
level = NOTSET
 
handlers = console
 

	
 
[logger_routes]
 
level = DEBUG
 
handlers = 
 
qualname = routes.middleware
 
# "level = DEBUG" logs the route matched and routing variables.
 
propagate = 1
 

	
 
[logger_beaker]
 
level = DEBUG
 
handlers = 
 
qualname = beaker.container
 
propagate = 1
 

	
 
[logger_templates]
 
level = INFO
 
handlers = 
 
qualname = pylons.templating
 
propagate = 1
 

	
 
[logger_rhodecode]
 
level = DEBUG
 
handlers = 
 
qualname = rhodecode
 
propagate = 1
 

	
 
[logger_sqlalchemy]
 
level = INFO
 
handlers = console_sql
 
qualname = sqlalchemy.engine
 
propagate = 0
 

	
 
[logger_whoosh_indexer]
 
level = DEBUG
 
handlers = 
 
qualname = whoosh_indexer
 
propagate = 1
 

	
 
##############
 
## HANDLERS ##
 
##############
 

	
 
[handler_console]
 
class = StreamHandler
 
args = (sys.stderr,)
 
level = DEBUG
 
formatter = color_formatter
 

	
 
[handler_console_sql]
 
class = StreamHandler
 
args = (sys.stderr,)
 
level = DEBUG
 
formatter = color_formatter_sql
 

	
 
################
 
## FORMATTERS ##
 
################
 

	
 
[formatter_generic]
 
format = %(asctime)s.%(msecs)03d %(levelname)-5.5s [%(name)s] %(message)s
 
datefmt = %Y-%m-%d %H:%M:%S
 

	
docs/api/api.rst
Show inline comments
 
@@ -6,622 +6,654 @@ API
 

	
 

	
 
Starting from RhodeCode version 1.2 a simple API was implemented.
 
There's a single schema for calling all api methods. API is implemented
 
with JSON protocol both ways. An url to send API request in RhodeCode is
 
<your_server>/_admin/api
 

	
 
API ACCESS FOR WEB VIEWS
 
++++++++++++++++++++++++
 

	
 
API access can also be turned on for each web view in RhodeCode that is 
 
decorated with `@LoginRequired` decorator. To enable API access simple change 
 
the standard login decorator to `@LoginRequired(api_access=True)`. 
 
After this change, a rhodecode view can be accessed without login by adding a 
 
GET parameter `?api_key=<api_key>` to url. By default this is only
 
enabled on RSS/ATOM feed views.
 

	
 

	
 
API ACCESS
 
++++++++++
 

	
 
All clients are required to send JSON-RPC spec JSON data::
 

	
 
    {   
 
        "id:<id>,
 
        "id:"<id>",
 
        "api_key":"<api_key>",
 
        "method":"<method_name>",
 
        "args":{"<arg_key>":"<arg_val>"}
 
    }
 

	
 
Example call for autopulling remotes repos using curl::
 
    curl https://server.com/_admin/api -X POST -H 'content-type:text/plain' --data-binary '{"id":1,"api_key":"xe7cdb2v278e4evbdf5vs04v832v0efvcbcve4a3","method":"pull","args":{"repo":"CPython"}}'
 

	
 
Simply provide
 
 - *id* A value of any type, which is used to match the response with the request that it is replying to.
 
 - *api_key* for access and permission validation.
 
 - *method* is name of method to call
 
 - *args* is an key:value list of arguments to pass to method
 

	
 
.. note::
 

	
 
    api_key can be found in your user account page
 

	
 

	
 
RhodeCode API will return always a JSON-RPC response::
 

	
 
    {   
 
        "id":<id>,
 
        "result": "<result>",
 
        "error": null
 
        "id":<id>, # matching id sent by request
 
        "result": "<result>"|null, # JSON formatted result, null if any errors
 
        "error": "null"|<error_message> # JSON formatted error (if any)
 
    }
 

	
 
All responses from API will be `HTTP/1.0 200 OK`, if there's an error while
 
calling api *error* key from response will contain failure description
 
and result will be null.
 

	
 
API METHODS
 
+++++++++++
 

	
 

	
 
pull
 
----
 

	
 
Pulls given repo from remote location. Can be used to automatically keep
 
remote repos up to date. This command can be executed only using api_key
 
belonging to user with admin rights
 

	
 
INPUT::
 

	
 
    id : <id_for_response>
 
    api_key : "<api_key>"
 
    method :  "pull"
 
    args :    {
 
                "repo_name" : "<reponame>"
 
              }
 

	
 
OUTPUT::
 

	
 
    result : "Pulled from <reponame>"
 
    error :  null
 

	
 

	
 
get_user
 
--------
 

	
 
Get's an user by username or user_id, Returns empty result if user is not found.
 
This command can be executed only using api_key belonging to user with admin 
 
rights.
 

	
 

	
 
INPUT::
 

	
 
    id : <id_for_response>
 
    api_key : "<api_key>"
 
    method :  "get_user"
 
    args :    { 
 
                "userid" : "<username or user_id>"
 
              }
 

	
 
OUTPUT::
 

	
 
    result: None if user does not exist or 
 
            {
 
                "id" :       "<id>",
 
                "username" : "<username>",
 
                "firstname": "<firstname>",
 
                "lastname" : "<lastname>",
 
                "email" :    "<email>",
 
                "active" :   "<bool>",
 
                "admin" :    "<bool>",
 
                "ldap_dn" :  "<ldap_dn>"
 
                "ldap_dn" :  "<ldap_dn>",
 
                "last_login": "<last_login>",
 
                "permissions": {
 
                    "global": ["hg.create.repository",
 
                               "repository.read",
 
                               "hg.register.manual_activate"],
 
                    "repositories": {"repo1": "repository.none"},
 
                    "repositories_groups": {"Group1": "group.read"}
 
                 },
 
            }
 

	
 
    error:  null
 

	
 

	
 
get_users
 
---------
 

	
 
Lists all existing users. This command can be executed only using api_key
 
belonging to user with admin rights.
 

	
 

	
 
INPUT::
 

	
 
    id : <id_for_response>
 
    api_key : "<api_key>"
 
    method :  "get_users"
 
    args :    { }
 

	
 
OUTPUT::
 

	
 
    result: [
 
              {
 
                "id" :       "<id>",
 
                "username" : "<username>",
 
                "firstname": "<firstname>",
 
                "lastname" : "<lastname>",
 
                "email" :    "<email>",
 
                "active" :   "<bool>",
 
                "admin" :    "<bool>",
 
                "ldap_dn" :  "<ldap_dn>"
 
                "ldap_dn" :  "<ldap_dn>",
 
                "last_login": "<last_login>",
 
              },
 
    	      …
 
            ]
 
    error:  null
 

	
 

	
 
create_user
 
-----------
 

	
 
Creates new user. This command can 
 
be executed only using api_key belonging to user with admin rights.
 

	
 

	
 
INPUT::
 

	
 
    id : <id_for_response>
 
    api_key : "<api_key>"
 
    method :  "create_user"
 
    args :    {
 
                "username" :  "<username>",
 
                "password" :  "<password>",
 
                "email" :     "<useremail>",
 
                "firstname" : "<firstname> = None",
 
                "lastname" :  "<lastname> = None",
 
                "active" :    "<bool> = True",
 
                "admin" :     "<bool> = False",
 
                "ldap_dn" :   "<ldap_dn> = None"
 
              }
 

	
 
OUTPUT::
 

	
 
    result: {
 
              "id" : "<new_user_id>",
 
              "msg" : "created new user <username>"
 
            }
 
    error:  null
 

	
 

	
 
update_user
 
-----------
 

	
 
updates current one if such user exists. This command can 
 
be executed only using api_key belonging to user with admin rights.
 

	
 

	
 
INPUT::
 

	
 
    id : <id_for_response>
 
    api_key : "<api_key>"
 
    method :  "update_user"
 
    args :    {
 
                "userid" : "<user_id or username>",
 
                "username" :  "<username>",
 
                "password" :  "<password>",
 
                "email" :     "<useremail>",
 
                "firstname" : "<firstname>",
 
                "lastname" :  "<lastname>",
 
                "active" :    "<bool>",
 
                "admin" :     "<bool>",
 
                "ldap_dn" :   "<ldap_dn>"
 
              }
 

	
 
OUTPUT::
 

	
 
    result: {
 
              "id" : "<edited_user_id>",
 
              "msg" : "updated user <username>"
 
            }
 
    error:  null
 

	
 

	
 
get_users_group
 
---------------
 

	
 
Gets an existing users group. This command can be executed only using api_key
 
belonging to user with admin rights.
 

	
 

	
 
INPUT::
 

	
 
    id : <id_for_response>
 
    api_key : "<api_key>"
 
    method :  "get_users_group"
 
    args :    {
 
                "group_name" : "<name>"
 
              }
 

	
 
OUTPUT::
 

	
 
    result : None if group not exist
 
             {
 
               "id" :         "<id>",
 
               "group_name" : "<groupname>",
 
               "active":      "<bool>",
 
               "members" :  [
 
                              { "id" :       "<userid>",
 
                                "username" : "<username>",
 
                                "firstname": "<firstname>",
 
                                "lastname" : "<lastname>",
 
                                "email" :    "<email>",
 
                                "active" :   "<bool>",
 
                                "admin" :    "<bool>",
 
                                "ldap" :     "<ldap_dn>"
 
                              },
 
                              …
 
                            ]
 
             }
 
    error : null
 

	
 

	
 
get_users_groups
 
----------------
 

	
 
Lists all existing users groups. This command can be executed only using 
 
api_key belonging to user with admin rights.
 

	
 

	
 
INPUT::
 

	
 
    id : <id_for_response>
 
    api_key : "<api_key>"
 
    method :  "get_users_groups"
 
    args :    { }
 

	
 
OUTPUT::
 

	
 
    result : [
 
               {
 
                 "id" :         "<id>",
 
                 "group_name" : "<groupname>",
 
                 "active":      "<bool>",
 
                 "members" :  [
 
	    	                    {
 
	    	                      "id" :       "<userid>",
 
	                              "username" : "<username>",
 
	                              "firstname": "<firstname>",
 
	                              "lastname" : "<lastname>",
 
	                              "email" :    "<email>",
 
	                              "active" :   "<bool>",
 
	                              "admin" :    "<bool>",
 
	                              "ldap" :     "<ldap_dn>"
 
	                            },
 
	    	                    …
 
	                          ]
 
	            }
 
              ]
 
    error : null
 

	
 

	
 
create_users_group
 
------------------
 

	
 
Creates new users group. This command can be executed only using api_key
 
belonging to user with admin rights
 

	
 

	
 
INPUT::
 

	
 
    id : <id_for_response>
 
    api_key : "<api_key>"
 
    method :  "create_users_group"
 
    args:     {
 
                "group_name":  "<groupname>",
 
                "active":"<bool> = True"
 
              }
 

	
 
OUTPUT::
 

	
 
    result: {
 
              "id":  "<newusersgroupid>",
 
              "msg": "created new users group <groupname>"
 
            }
 
    error:  null
 

	
 

	
 
add_user_to_users_group
 
-----------------------
 

	
 
Adds a user to a users group. If user exists in that group success will be 
 
`false`. This command can be executed only using api_key
 
belonging to user with admin rights
 

	
 

	
 
INPUT::
 

	
 
    id : <id_for_response>
 
    api_key : "<api_key>"
 
    method :  "add_user_users_group"
 
    args:     {
 
                "group_name" :  "<groupname>",
 
                "username" :   "<username>"
 
              }
 

	
 
OUTPUT::
 

	
 
    result: {
 
              "id":  "<newusersgroupmemberid>",
 
              "success": True|False # depends on if member is in group
 
              "msg": "added member <username> to users group <groupname> | 
 
                      User is already in that group"
 
            }
 
    error:  null
 

	
 

	
 
remove_user_from_users_group
 
----------------------------
 

	
 
Removes a user from a users group. If user is not in given group success will
 
be `false`. This command can be executed only 
 
using api_key belonging to user with admin rights
 

	
 

	
 
INPUT::
 

	
 
    id : <id_for_response>
 
    api_key : "<api_key>"
 
    method :  "remove_user_from_users_group"
 
    args:     {
 
                "group_name" :  "<groupname>",
 
                "username" :   "<username>"
 
              }
 

	
 
OUTPUT::
 

	
 
    result: {
 
              "success":  True|False,  # depends on if member is in group
 
              "msg": "removed member <username> from users group <groupname> | 
 
                      User wasn't in group"
 
            }
 
    error:  null
 

	
 

	
 
get_repo
 
--------
 

	
 
Gets an existing repository by it's name or repository_id. This command can 
 
Gets an existing repository by it's name or repository_id. Members will return
 
either users_group or user associated to that repository. This command can 
 
be executed only using api_key belonging to user with admin rights.
 

	
 

	
 
INPUT::
 

	
 
    id : <id_for_response>
 
    api_key : "<api_key>"
 
    method :  "get_repo"
 
    args:     {
 
                "repoid" : "<reponame or repo_id>"
 
              }
 

	
 
OUTPUT::
 

	
 
    result: None if repository does not exist or
 
            {
 
                "id" :          "<id>",
 
                "repo_name" :   "<reponame>"
 
                "type" :        "<type>",
 
                "description" : "<description>",
 
                "members" :     [
 
                                  { "id" :         "<userid>",
 
                                  { 
 
                                    "type": "user",
 
                                    "id" :         "<userid>",
 
                                    "username" :   "<username>",
 
                                    "firstname":   "<firstname>",
 
                                    "lastname" :   "<lastname>",
 
                                    "email" :      "<email>",
 
                                    "active" :     "<bool>",
 
                                    "admin" :      "<bool>",
 
                                    "ldap" :       "<ldap_dn>",
 
                                    "permission" : "repository.(read|write|admin)"
 
                                  },
 
                                  …
 
                                  {
 
                                  { 
 
                                    "type": "users_group",
 
                                    "id" :       "<usersgroupid>",
 
                                    "name" :     "<usersgroupname>",
 
                                    "active":    "<bool>",
 
                                    "permission" : "repository.(read|write|admin)"
 
                                  },
 
                                  …
 
                                ]
 
            }
 
    error:  null
 

	
 

	
 
get_repos
 
---------
 

	
 
Lists all existing repositories. This command can be executed only using api_key
 
belonging to user with admin rights
 

	
 

	
 
INPUT::
 

	
 
    id : <id_for_response>
 
    api_key : "<api_key>"
 
    method :  "get_repos"
 
    args:     { }
 

	
 
OUTPUT::
 

	
 
    result: [
 
              {
 
                "id" :          "<id>",
 
                "repo_name" :   "<reponame>"
 
                "type" :        "<type>",
 
                "description" : "<description>"
 
              },
 
              …
 
            ]
 
    error:  null
 

	
 

	
 
get_repo_nodes
 
--------------
 

	
 
returns a list of nodes and it's children in a flat list for a given path 
 
at given revision. It's possible to specify ret_type to show only `files` or 
 
`dirs`. This command can be executed only using api_key belonging to user 
 
with admin rights
 

	
 

	
 
INPUT::
 

	
 
    id : <id_for_response>
 
    api_key : "<api_key>"
 
    method :  "get_repo_nodes"
 
    args:     {
 
                "repo_name" : "<reponame>",
 
                "revision"  : "<revision>",
 
                "root_path" : "<root_path>",
 
                "ret_type"  : "<ret_type>" = 'all'
 
              }
 

	
 
OUTPUT::
 

	
 
    result: [
 
              {
 
                "name" :        "<name>"
 
                "type" :        "<type>",
 
              },
 
              …
 
            ]
 
    error:  null
 

	
 

	
 
create_repo
 
-----------
 

	
 
Creates a repository. This command can be executed only using api_key
 
belonging to user with admin rights.
 
If repository name contains "/", all needed repository groups will be created.
 
For example "foo/bar/baz" will create groups "foo", "bar" (with "foo" as parent),
 
and create "baz" repository with "bar" as group.
 

	
 

	
 
INPUT::
 

	
 
    id : <id_for_response>
 
    api_key : "<api_key>"
 
    method :  "create_repo"
 
    args:     {
 
                "repo_name" :   "<reponame>",
 
                "owner_name" :  "<ownername>",
 
                "description" : "<description> = ''",
 
                "repo_type" :   "<type> = 'hg'",
 
                "private" :     "<bool> = False",
 
                "clone_uri" :   "<clone_uri> = None",
 
              }
 

	
 
OUTPUT::
 

	
 
    result: {
 
              "id": "<newrepoid>",
 
              "msg": "Created new repository <reponame>",
 
            }
 
    error:  null
 

	
 

	
 
delete_repo
 
-----------
 

	
 
Deletes a repository. This command can be executed only using api_key
 
belonging to user with admin rights.
 

	
 

	
 
INPUT::
 

	
 
    id : <id_for_response>
 
    api_key : "<api_key>"
 
    method :  "delete_repo"
 
    args:     {
 
                "repo_name" :   "<reponame>",
 
              }
 

	
 
OUTPUT::
 

	
 
    result: {
 
              "msg": "Deleted repository <reponame>",
 
            }
 
    error:  null
 

	
 

	
 
grant_user_permission
 
---------------------
 

	
 
Grant permission for user on given repository, or update existing one
 
if found. This command can be executed only using api_key belonging to user 
 
with admin rights.
 

	
 

	
 
INPUT::
 

	
 
    id : <id_for_response>
 
    api_key : "<api_key>"
 
    method :  "grant_user_permission"
 
    args:     {
 
                "repo_name" :  "<reponame>",
 
                "username" :   "<username>",
 
                "perm" :       "(repository.(none|read|write|admin))",
 
              }
 

	
 
OUTPUT::
 

	
 
    result: {
 
              "msg" : "Granted perm: <perm> for user: <username> in repo: <reponame>"
 
            }
 
    error:  null
 

	
 

	
 
revoke_user_permission
 
----------------------
 

	
 
Revoke permission for user on given repository. This command can be executed 
 
only using api_key belonging to user with admin rights.
 

	
 

	
 
INPUT::
 

	
 
    id : <id_for_response>
 
    api_key : "<api_key>"
 
    method  : "revoke_user_permission"
 
    args:     {
 
                "repo_name" :  "<reponame>",
 
                "username" :   "<username>",
 
              }
 

	
 
OUTPUT::
 

	
 
    result: {
 
              "msg" : "Revoked perm for user: <suername> in repo: <reponame>"
 
            }
 
    error:  null
 

	
 

	
 
grant_users_group_permission
 
----------------------------
 

	
 
Grant permission for users group on given repository, or update
 
existing one if found. This command can be executed only using 
 
api_key belonging to user with admin rights.
 

	
 

	
 
INPUT::
 

	
 
    id : <id_for_response>
 
    api_key : "<api_key>"
 
    method :  "grant_users_group_permission"
 
    args:     {
 
                "repo_name" : "<reponame>",
 
                "group_name" : "<usersgroupname>",
 
                "perm" : "(repository.(none|read|write|admin))",
 
              }
 

	
 
OUTPUT::
 

	
 
    result: {
 
              "msg" : "Granted perm: <perm> for group: <usersgroupname> in repo: <reponame>"
 
            }
 
    error:  null
 
    
 
    
 
revoke_users_group_permission
 
-----------------------------
 

	
 
Revoke permission for users group on given repository.This command can be 
 
executed only using api_key belonging to user with admin rights.
 

	
 
INPUT::
 

	
 
    id : <id_for_response>
 
    api_key : "<api_key>"
 
    method  : "revoke_users_group_permission"
 
    args:     {
 
                "repo_name" :  "<reponame>",
 
                "users_group" :   "<usersgroupname>",
 
              }
 

	
 
OUTPUT::
 

	
 
    result: {
 
              "msg" : "Revoked perm for group: <usersgroupname> in repo: <reponame>"
 
            }
 
    error:  null
 
\ No newline at end of file
docs/changelog.rst
Show inline comments
 
.. _changelog:
 

	
 
=========
 
Changelog
 
=========
 

	
 

	
 
1.3.4 (**2012-03-28**)
 
----------------------
 

	
 
news
 
++++
 

	
 
- Whoosh logging is now controlled by the .ini files logging setup
 
- added clone-url into edit form on /settings page
 
- added help text into repo add/edit forms
 
- created rcextensions module with additional mappings (ref #322) and
 
  post push/pull/create repo hooks callbacks
 
- implemented #377 Users view for his own permissions on account page
 
- #399 added inheritance of permissions for users group on repos groups
 
- #401 repository group is automatically pre-selected when adding repos 
 
  inside a repository group
 
- added alternative HTTP 403 response when client failed to authenticate. Helps 
 
  solving issues with Mercurial and LDAP
 
- #402 removed group prefix from repository name when listing repositories 
 
  inside a group
 
- added gravatars into permission view and permissions autocomplete
 
- #347 when running multiple RhodeCode instances, properly invalidates cache 
 
  for all registered servers
 

	
 
fixes
 
+++++
 

	
 
- fixed #390 cache invalidation problems on repos inside group
 
- fixed #385 clone by ID url was loosing proxy prefix in URL
 
- fixed some unicode problems with waitress
 
- fixed issue with escaping < and > in changeset commits
 
- fixed error occurring during recursive group creation in API 
 
  create_repo function
 
- fixed #393 py2.5 fixes for routes url generator
 
- fixed #397 Private repository groups shows up before login
 
- fixed #396 fixed problems with revoking users in nested groups
 
- fixed mysql unicode issues + specified InnoDB as default engine with 
 
  utf8 charset
 
- #406 trim long branch/tag names in changelog to not break UI
 
  
 
1.3.3 (**2012-03-02**)
 
----------------------
 

	
 
news
 
++++
 

	
 

	
 
fixes
 
+++++
 

	
 
- fixed some python2.5 compatibility issues 
 
- fixed issues with removed repos was accidentally added as groups, after
 
  full rescan of paths
 
- fixes #376 Cannot edit user (using container auth)
 
- fixes #378 Invalid image urls on changeset screen with proxy-prefix 
 
  configuration
 
- fixed initial sorting of repos inside repo group
 
- fixes issue when user tried to resubmit same permission into user/user_groups
 
- bumped beaker version that fixes #375 leap error bug
 
- fixed raw_changeset for git. It was generated with hg patch headers
 
- fixed vcs issue with last_changeset for filenodes
 
- fixed missing commit after hook delete
 
- fixed #372 issues with git operation detection that caused a security issue 
 
  for git repos
docs/index.rst
Show inline comments
 
@@ -2,49 +2,50 @@
 

	
 
.. include:: ./../README.rst
 

	
 
Users Guide
 
-----------
 

	
 
**Installation:**
 

	
 
.. toctree::
 
   :maxdepth: 1
 

	
 
   installation
 
   setup
 
   upgrade
 
   
 
**Usage**
 

	
 
.. toctree::
 
   :maxdepth: 1
 

	
 
   usage/general
 
   usage/git_support
 
   usage/statistics
 
   usage/backup
 
   
 
   usage/debugging
 

	
 
**Develop**
 

	
 
.. toctree::
 
   :maxdepth: 1
 
   
 
   contributing
 
   changelog
 

	
 
**API**
 

	
 
.. toctree::
 
   :maxdepth: 1
 

	
 
   api/api
 
   api/models
 
   
 

	
 
Other topics
 
------------
 

	
 
* :ref:`genindex`
 
* :ref:`search`
 

	
 
.. _virtualenv: http://pypi.python.org/pypi/virtualenv
docs/setup.rst
Show inline comments
 
.. _setup:
 

	
 
=====
 
Setup
 
=====
 

	
 

	
 
Setting up RhodeCode
 
--------------------
 

	
 
First, you will need to create a RhodeCode configuration file. Run the 
 
following command to do this::
 
 
 
    paster make-config RhodeCode production.ini
 

	
 
- This will create the file `production.ini` in the current directory. This
 
  configuration file contains the various settings for RhodeCode, e.g proxy 
 
  port, email settings, usage of static files, cache, celery settings and 
 
  logging.
 

	
 

	
 
Next, you need to create the databases used by RhodeCode. I recommend that you
 
use sqlite (default) or postgresql. If you choose a database other than the
 
use postgresql or sqlite (default). If you choose a database other than the
 
default ensure you properly adjust the db url in your production.ini
 
configuration file to use this other database. Create the databases by running
 
configuration file to use this other database. RhodeCode currently supports
 
postgresql, sqlite and mysql databases. Create the database by running
 
the following command::
 

	
 
    paster setup-app production.ini
 

	
 
This will prompt you for a "root" path. This "root" path is the location where
 
RhodeCode will store all of its repositories on the current machine. After
 
entering this "root" path ``setup-app`` will also prompt you for a username 
 
and password for the initial admin account which ``setup-app`` sets up for you.
 

	
 
- The ``setup-app`` command will create all of the needed tables and an admin
 
  account. When choosing a root path you can either use a new empty location, 
 
  or a location which already contains existing repositories. If you choose a 
 
  location which contains existing repositories RhodeCode will simply add all 
 
  of the repositories at the chosen location to it's database. (Note: make 
 
  sure you specify the correct path to the root).
 
- Note: the given path for mercurial_ repositories **must** be write accessible
 
  for the application. It's very important since the RhodeCode web interface 
 
  will work without write access, but when trying to do a push it will 
 
  eventually fail with permission denied errors unless it has write access.
 

	
 
You are now ready to use RhodeCode, to run it simply execute::
 
 
 
    paster serve production.ini
 
 
 
- This command runs the RhodeCode server. The web app should be available at the 
 
  127.0.0.1:5000. This ip and port is configurable via the production.ini 
 
  file created in previous step
 
- Use the admin account you created above when running ``setup-app`` to login 
 
  to the web app.
 
- The default permissions on each repository is read, and the owner is admin. 
 
  Remember to update these if needed.
 
- In the admin panel you can toggle ldap, anonymous, permissions settings. As
 
  well as edit more advanced options on users and repositories
 

	
 
Try copying your own mercurial repository into the "root" directory you are
 
using, then from within the RhodeCode web application choose Admin >
 
repositories. Then choose Add New Repository. Add the repository you copied 
 
into the root. Test that you can browse your repository from within RhodeCode 
 
and then try cloning your repository from RhodeCode with::
 
Optionally users can create `rcextensions` package that extends RhodeCode
 
functionality. To do this simply execute::
 

	
 
    paster make-rcext production.ini
 

	
 
    hg clone http://127.0.0.1:5000/<repository name>
 
This will create `rcextensions` package in the same place that your `ini` file
 
lives. With `rcextensions` it's possible to add additional mapping for whoosh, 
 
stats and add additional code into the push/pull/create repo hooks. For example
 
for sending signals to build-bots such as jenkins.
 
Please see the `__init__.py` file inside `rcextensions` package 
 
for more details.
 

	
 
where *repository name* is replaced by the name of your repository.
 

	
 
Using RhodeCode with SSH
 
------------------------
 

	
 
RhodeCode currently only hosts repositories using http and https. (The addition
 
of ssh hosting is a planned future feature.) However you can easily use ssh in
 
parallel with RhodeCode. (Repository access via ssh is a standard "out of
 
the box" feature of mercurial_ and you can use this to access any of the
 
repositories that RhodeCode is hosting. See PublishingRepositories_)
 

	
 
RhodeCode repository structures are kept in directories with the same name 
 
as the project. When using repository groups, each group is a subdirectory.
 
This allows you to easily use ssh for accessing repositories.
 

	
 
In order to use ssh you need to make sure that your web-server and the users 
 
login accounts have the correct permissions set on the appropriate directories.
 
(Note that these permissions are independent of any permissions you have set up
 
using the RhodeCode web interface.)
 

	
 
If your main directory (the same as set in RhodeCode settings) is for example
 
set to **/home/hg** and the repository you are using is named `rhodecode`, then
 
to clone via ssh you should run::
 

	
 
    hg clone ssh://user@server.com/home/hg/rhodecode
docs/upgrade.rst
Show inline comments
 
@@ -26,30 +26,33 @@ Then make sure you run the following com
 
 
 
This will display any changes made by the new version of RhodeCode to your
 
current configuration. It will try to perform an automerge. It's always better
 
to make a backup of your configuration file before hand and re check the 
 
content after the automerge.
 

	
 
.. note::
 
   Please always make sure your .ini files are up to date. Often errors are
 
   caused by missing params added in new versions.
 

	
 

	
 
It is also recommended that you rebuild the whoosh index after upgrading since 
 
the new whoosh version could introduce some incompatible index changes. Please
 
Read the changelog to see if there were any changes to whoosh.
 

	
 

	
 
The final step is to upgrade the database. To do this simply run::
 

	
 
    paster upgrade-db production.ini
 
 
 
This will upgrade the schema and update some of the defaults in the database,
 
and will always recheck the settings of the application, if there are no new 
 
options that need to be set.
 

	
 
.. note::
 
   If you're using Celery, make sure you restart all instances of it after
 
   upgrade.
 

	
 
.. _virtualenv: http://pypi.python.org/pypi/virtualenv  
 
.. _python: http://www.python.org/
 
.. _mercurial: http://mercurial.selenic.com/
 
.. _celery: http://celeryproject.org/
 
.. _rabbitmq: http://www.rabbitmq.com/
 
\ No newline at end of file
docs/usage/debugging.rst
Show inline comments
 
new file 100644
 
.. _debugging:
 

	
 
===================
 
Debugging RhodeCode
 
===================
 

	
 
If you encountered problems with RhodeCode here are some instructions how to
 
possibly debug them.
 

	
 
** First make sure you're using the latest version available.**
 

	
 
enable detailed debug
 
---------------------
 

	
 
RhodeCode uses standard python logging modules to log it's output.
 
By default only loggers with INFO level are displayed. To enable full output
 
change `level = DEBUG` for all logging handlers in currently used .ini file. 
 
This change will allow to see much more detailed output in the logfile or
 
console. This generally helps a lot to track issues.
 

	
 

	
 
enable interactive debug mode
 
-----------------------------
 

	
 
To enable interactive debug mode simply comment out `set debug = false` in
 
.ini file, this will trigger and interactive debugger each time there an
 
error in browser, or send a http link if error occured in the backend. This
 
is a great tool for fast debugging as you get a handy python console right
 
in the web view. ** NEVER ENABLE THIS ON PRODUCTION ** the interactive console
 
can be a serious security threat to you system.
docs/usage/general.rst
Show inline comments
 
@@ -50,31 +50,36 @@ example::
 
  # after insertion to test_group group the url will be
 
  http://server.com/test_group/repo_name
 
  
 
This can be an issue for build systems and any other hardcoded scripts, moving
 
repository to a group leads to a need for changing external systems. To 
 
overcome this RhodeCode introduces a non changable replacement url. It's 
 
simply an repository ID prefixed with `_` above urls are also accessible as::
 

	
 
  http://server.com/_<ID>
 
  
 
Since ID are always the same moving the repository will not affect such url.
 
the _<ID> syntax can be used anywhere in the system so urls with repo_name 
 
for changelogs, files and other can be exchanged with _<ID> syntax.
 

	
 

	
 

	
 
Mailing
 
-------
 

	
 
When administrator will fill up the mailing settings in .ini files
 
RhodeCode will send mails on user registration, or when RhodeCode errors occur
 
on errors the mails will have a detailed traceback of error.
 

	
 

	
 
Mails are also sent for code comments. If someone comments on a changeset
 
mail is sent to all participants, the person who commited the changeset 
 
(if present in RhodeCode), and to all people mentioned with @mention system.
 

	
 

	
 
Trending source files
 
---------------------
 

	
 
Trending source files are calculated based on pre defined dict of known
 
types and extensions. If You miss some extension or Would like to scan some
 
custom files it's possible to add new types in `LANGUAGES_EXTENSIONS_MAP` dict
 
located in `/rhodecode/lib/celerylib/tasks.py`
 
\ No newline at end of file
init.d/rhodecode-daemon4
Show inline comments
 
new file 100644
 
#!/bin/bash
 
###########################################
 
#### THIS IS AN ARCH LINUX RC.D SCRIPT ####
 
###########################################
 

	
 
. /etc/rc.conf
 
. /etc/rc.d/functions
 

	
 
DAEMON=rhodecode
 
APP_HOMEDIR="/srv"
 
APP_PATH="$APP_HOMEDIR/$DAEMON"
 
CONF_NAME="production.ini"
 
LOG_FILE="/var/log/$DAEMON.log"
 
PID_FILE="/run/daemons/$DAEMON"
 
APPL=/usr/bin/paster
 
RUN_AS="*****"
 

	
 
ARGS="serve --daemon \
 
--user=$RUN_AS \
 
--group=$RUN_AS \
 
--pid-file=$PID_FILE \
 
--log-file=$LOG_FILE \
 
$APP_PATH/$CONF_NAME"
 

	
 
[ -r /etc/conf.d/$DAEMON ] && . /etc/conf.d/$DAEMON
 

	
 
if [[ -r $PID_FILE ]]; then
 
    read -r PID < "$PID_FILE"
 
    if [[ $PID && ! -d /proc/$PID ]]; then
 
        unset PID
 
        rm_daemon $DAEMON
 
    fi
 
fi
 

	
 
case "$1" in
 
start)
 
    stat_busy "Starting $DAEMON"
 
    export HOME=$APP_PATH
 
    [ -z "$PID" ] && $APPL $ARGS &>/dev/null
 
    if [ $? = 0 ]; then
 
        add_daemon $DAEMON
 
        stat_done
 
    else
 
        stat_fail
 
        exit 1
 
    fi
 
    ;;
 
stop)
 
    stat_busy "Stopping $DAEMON"
 
    [ -n "$PID" ] && kill $PID &>/dev/null 
 
    if [ $? = 0 ]; then
 
        rm_daemon $DAEMON
 
        stat_done
 
    else
 
        stat_fail
 
        exit 1
 
    fi
 
    ;;
 
restart)
 
    $0 stop
 
    sleep 1
 
    $0 start
 
    ;;
 
status)
 
    stat_busy "Checking $name status";
 
    ck_status $name
 
    ;;
 
*)
 
    echo "usage: $0 {start|stop|restart|status}"
 
esac
 
\ No newline at end of file
production.ini
Show inline comments
 
@@ -72,48 +72,53 @@ default_encoding = utf8
 
## comment out issue_pat, issue_server, issue_prefix to enable
 

	
 
## pattern to get the issues from commit messages
 
## default one used here is #<numbers> with a regex passive group for `#`
 
## {id} will be all groups matched from this pattern
 

	
 
issue_pat = (?:\s*#)(\d+)
 

	
 
## server url to the issue, each {id} will be replaced with match
 
## fetched from the regex and {repo} is replaced with repository name
 

	
 
issue_server_link = https://myissueserver.com/{repo}/issue/{id}
 

	
 
## prefix to add to link to indicate it's an url
 
## #314 will be replaced by <issue_prefix><id>
 

	
 
issue_prefix = #
 

	
 
## instance-id prefix
 
## a prefix key for this instance used for cache invalidation when running 
 
## multiple instances of rhodecode, make sure it's globally unique for 
 
## all running rhodecode instances. Leave empty if you don't use it
 
instance_id = 
 

	
 
## alternative return HTTP header for failed authentication. Default HTTP
 
## response is 401 HTTPUnauthorized. Currently HG clients have troubles with 
 
## handling that. Set this variable to 403 to return HTTPForbidden
 
auth_ret_code =
 

	
 
####################################
 
###        CELERY CONFIG        ####
 
####################################
 
use_celery = false
 
broker.host = localhost
 
broker.vhost = rabbitmqhost
 
broker.port = 5672
 
broker.user = rabbitmq
 
broker.password = qweqwe
 

	
 
celery.imports = rhodecode.lib.celerylib.tasks
 

	
 
celery.result.backend = amqp
 
celery.result.dburi = amqp://
 
celery.result.serialier = json
 

	
 
#celery.send.task.error.emails = true
 
#celery.amqp.task.result.expires = 18000
 

	
 
celeryd.concurrency = 2
 
#celeryd.log.file = celeryd.log
 
celeryd.log.level = debug
 
celeryd.max.tasks.per.child = 1
 

	
 
@@ -187,100 +192,106 @@ beaker.session.auto = False
 
#beaker.session.cookie_expires = 3600
 

	
 

	
 
################################################################################
 
## WARNING: *THE LINE BELOW MUST BE UNCOMMENTED ON A PRODUCTION ENVIRONMENT*  ##
 
## Debug mode will enable the interactive debugging tool, allowing ANYONE to  ##
 
## execute malicious code after an exception is raised.                       ##
 
################################################################################
 
set debug = false
 

	
 
##################################
 
###       LOGVIEW CONFIG       ###
 
##################################
 
logview.sqlalchemy = #faa
 
logview.pylons.templating = #bfb
 
logview.pylons.util = #eee
 

	
 
#########################################################
 
### DB CONFIGS - EACH DB WILL HAVE IT'S OWN CONFIG    ###
 
#########################################################
 
#sqlalchemy.db1.url = sqlite:///%(here)s/rhodecode.db
 
sqlalchemy.db1.url = postgresql://postgres:qwe@localhost/rhodecode
 
sqlalchemy.db1.echo = false
 
sqlalchemy.db1.pool_recycle = 3600
 
sqlalchemy.convert_unicode = true
 
sqlalchemy.db1.convert_unicode = true
 

	
 
################################
 
### LOGGING CONFIGURATION   ####
 
################################
 
[loggers]
 
keys = root, routes, rhodecode, sqlalchemy, beaker, templates
 
keys = root, routes, rhodecode, sqlalchemy, beaker, templates, whoosh_indexer
 

	
 
[handlers]
 
keys = console, console_sql
 

	
 
[formatters]
 
keys = generic, color_formatter, color_formatter_sql
 

	
 
#############
 
## LOGGERS ##
 
#############
 
[logger_root]
 
level = NOTSET
 
handlers = console
 

	
 
[logger_routes]
 
level = DEBUG
 
handlers = 
 
qualname = routes.middleware
 
# "level = DEBUG" logs the route matched and routing variables.
 
propagate = 1
 

	
 
[logger_beaker]
 
level = DEBUG
 
handlers = 
 
qualname = beaker.container
 
propagate = 1
 

	
 
[logger_templates]
 
level = INFO
 
handlers = 
 
qualname = pylons.templating
 
propagate = 1
 

	
 
[logger_rhodecode]
 
level = DEBUG
 
handlers = 
 
qualname = rhodecode
 
propagate = 1
 

	
 
[logger_sqlalchemy]
 
level = INFO
 
handlers = console_sql
 
qualname = sqlalchemy.engine
 
propagate = 0
 

	
 
[logger_whoosh_indexer]
 
level = DEBUG
 
handlers = 
 
qualname = whoosh_indexer
 
propagate = 1
 

	
 
##############
 
## HANDLERS ##
 
##############
 

	
 
[handler_console]
 
class = StreamHandler
 
args = (sys.stderr,)
 
level = INFO
 
formatter = generic
 

	
 
[handler_console_sql]
 
class = StreamHandler
 
args = (sys.stderr,)
 
level = WARN
 
formatter = generic
 

	
 
################
 
## FORMATTERS ##
 
################
 

	
 
[formatter_generic]
 
format = %(asctime)s.%(msecs)03d %(levelname)-5.5s [%(name)s] %(message)s
 
datefmt = %Y-%m-%d %H:%M:%S
 

	
requires.txt
Show inline comments
 
Pylons==1.0.0
 
Beaker==1.6.3
 
WebHelpers>=1.2
 
WebHelpers==1.3
 
formencode==1.2.4
 
SQLAlchemy==0.7.4
 
Mako==0.5.0
 
SQLAlchemy==0.7.6
 
Mako==0.6.2
 
pygments>=1.4
 
whoosh>=2.3.0,<2.4
 
celery>=2.2.5,<2.3
 
babel
 
python-dateutil>=1.5.0,<2.0.0
 
dulwich>=0.8.0,<0.9.0
 
https://github.com/jelmer/dulwich/tarball/master
 
webob==1.0.8
 
markdown==2.1.1
 
docutils==0.8.1
 
py-bcrypt
 
mercurial>=2.1,<2.2
 
\ No newline at end of file
 
mercurial>=2.1,<2.2
rhodecode/__init__.py
Show inline comments
 
# -*- coding: utf-8 -*-
 
"""
 
    rhodecode.__init__
 
    ~~~~~~~~~~~~~~~~~~
 

	
 
    RhodeCode, a web based repository management based on pylons
 
    versioning implementation: http://semver.org/
 
    versioning implementation: http://www.python.org/dev/peps/pep-0386/
 

	
 
    :created_on: Apr 9, 2010
 
    :author: marcink
 
    :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>
 
    :license: GPLv3, see COPYING for more details.
 
"""
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
import sys
 
import platform
 

	
 
VERSION = (1, 3, 3)
 
__version__ = '.'.join((str(each) for each in VERSION[:4]))
 
VERSION = (1, 3, 4)
 

	
 
try:
 
    from rhodecode.lib import get_current_revision
 
    _rev = get_current_revision()
 
    if _rev:
 
        VERSION += ('dev%s' % _rev[0],)
 
except ImportError:
 
    pass
 

	
 
__version__ = ('.'.join((str(each) for each in VERSION[:3])) +
 
               '.'.join(VERSION[3:]))
 
__dbversion__ = 5  # defines current db version for migrations
 
__platform__ = platform.system()
 
__license__ = 'GPLv3'
 
__py_version__ = sys.version_info
 

	
 
PLATFORM_WIN = ('Windows')
 
PLATFORM_OTHERS = ('Linux', 'Darwin', 'FreeBSD', 'OpenBSD', 'SunOS')
 

	
 
requirements = [
 
    "Pylons==1.0.0",
 
    "Beaker==1.6.3",
 
    "WebHelpers>=1.2",
 
    "WebHelpers==1.3",
 
    "formencode==1.2.4",
 
    "SQLAlchemy==0.7.4",
 
    "Mako==0.5.0",
 
    "SQLAlchemy==0.7.6",
 
    "Mako==0.6.2",
 
    "pygments>=1.4",
 
    "whoosh>=2.3.0,<2.4",
 
    "celery>=2.2.5,<2.3",
 
    "babel",
 
    "python-dateutil>=1.5.0,<2.0.0",
 
    "dulwich>=0.8.0,<0.9.0",
 
    "dulwich>=0.8.4,<0.9.0",
 
    "webob==1.0.8",
 
    "markdown==2.1.1",
 
    "docutils==0.8.1",
 
]
 

	
 
if __py_version__ < (2, 6):
 
    requirements.append("simplejson")
 
    requirements.append("pysqlite")
 

	
 
if __platform__ in PLATFORM_WIN:
 
    requirements.append("mercurial>=2.1,<2.2")
 
else:
 
    requirements.append("py-bcrypt")
 
    requirements.append("mercurial>=2.1,<2.2")
 

	
 

	
 
try:
 
    from rhodecode.lib import get_current_revision
 
    _rev = get_current_revision(quiet=True)
 
except ImportError:
 
    # this is needed when doing some setup.py operations
 
    _rev = False
 

	
 
if len(VERSION) > 3 and _rev:
 
    __version__ += ' [rev:%s]' % _rev[0]
 

	
 

	
 
def get_version():
 
    """Returns shorter version (digit parts only) as string."""
 

	
 
    return '.'.join((str(each) for each in VERSION[:3]))
 

	
 
BACKENDS = {
 
    'hg': 'Mercurial repository',
 
    'git': 'Git repository',
 
}
 

	
 
CELERY_ON = False
 

	
 
# link to config for pylons
 
CONFIG = {}
 

	
 
# Linked module for extensions
 
EXTENSIONS = {}
rhodecode/config/conf.py
Show inline comments
 
new file 100644
 
# -*- coding: utf-8 -*-
 
"""
 
    package.rhodecode.config.conf
 
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
    Various config settings for RhodeCode
 

	
 
    :created_on: Mar 7, 2012
 
    :author: marcink
 
    :copyright: (C) 2009-2010 Marcin Kuzminski <marcin@python-works.com>
 
    :license: <name>, see LICENSE_FILE for more details.
 
"""
 
from rhodecode import EXTENSIONS
 

	
 
from rhodecode.lib.utils2 import __get_lem
 

	
 

	
 
# language map is also used by whoosh indexer, which for those specified
 
# extensions will index it's content
 
LANGUAGES_EXTENSIONS_MAP = __get_lem()
 

	
 
#==============================================================================
 
# WHOOSH INDEX EXTENSIONS
 
#==============================================================================
 
# EXTENSIONS WE WANT TO INDEX CONTENT OFF USING WHOOSH
 
INDEX_EXTENSIONS = LANGUAGES_EXTENSIONS_MAP.keys()
 

	
 
# list of readme files to search in file tree and display in summary
 
# attached weights defines the search  order lower is first
 
ALL_READMES = [
 
    ('readme', 0), ('README', 0), ('Readme', 0),
 
    ('doc/readme', 1), ('doc/README', 1), ('doc/Readme', 1),
 
    ('Docs/readme', 2), ('Docs/README', 2), ('Docs/Readme', 2),
 
    ('DOCS/readme', 2), ('DOCS/README', 2), ('DOCS/Readme', 2),
 
    ('docs/readme', 2), ('docs/README', 2), ('docs/Readme', 2),
 
]
 

	
 
# extension together with weights to search lower is first
 
RST_EXTS = [
 
    ('', 0), ('.rst', 1), ('.rest', 1),
 
    ('.RST', 2), ('.REST', 2),
 
    ('.txt', 3), ('.TXT', 3)
 
]
 

	
 
MARKDOWN_EXTS = [
 
    ('.md', 1), ('.MD', 1),
 
    ('.mkdn', 2), ('.MKDN', 2),
 
    ('.mdown', 3), ('.MDOWN', 3),
 
    ('.markdown', 4), ('.MARKDOWN', 4)
 
]
 

	
 
PLAIN_EXTS = [('.text', 2), ('.TEXT', 2)]
 

	
 
ALL_EXTS = MARKDOWN_EXTS + RST_EXTS + PLAIN_EXTS
 

	
 
DATETIME_FORMAT = "%Y-%m-%d %H:%M:%S"
 

	
 
DATE_FORMAT = "%Y-%m-%d"
rhodecode/config/deployment.ini_tmpl
Show inline comments
 
@@ -72,48 +72,53 @@ default_encoding = utf8
 
## comment out issue_pat, issue_server, issue_prefix to enable
 

	
 
## pattern to get the issues from commit messages
 
## default one used here is #<numbers> with a regex passive group for `#`
 
## {id} will be all groups matched from this pattern
 

	
 
issue_pat = (?:\s*#)(\d+)
 

	
 
## server url to the issue, each {id} will be replaced with match
 
## fetched from the regex and {repo} is replaced with repository name
 

	
 
issue_server_link = https://myissueserver.com/{repo}/issue/{id}
 

	
 
## prefix to add to link to indicate it's an url
 
## #314 will be replaced by <issue_prefix><id>
 

	
 
issue_prefix = #
 

	
 
## instance-id prefix
 
## a prefix key for this instance used for cache invalidation when running 
 
## multiple instances of rhodecode, make sure it's globally unique for 
 
## all running rhodecode instances. Leave empty if you don't use it
 
instance_id = 
 

	
 
## alternative return HTTP header for failed authentication. Default HTTP
 
## response is 401 HTTPUnauthorized. Currently HG clients have troubles with 
 
## handling that. Set this variable to 403 to return HTTPForbidden
 
auth_ret_code =
 

	
 
####################################
 
###        CELERY CONFIG        ####
 
####################################
 
use_celery = false
 
broker.host = localhost
 
broker.vhost = rabbitmqhost
 
broker.port = 5672
 
broker.user = rabbitmq
 
broker.password = qweqwe
 

	
 
celery.imports = rhodecode.lib.celerylib.tasks
 

	
 
celery.result.backend = amqp
 
celery.result.dburi = amqp://
 
celery.result.serialier = json
 

	
 
#celery.send.task.error.emails = true
 
#celery.amqp.task.result.expires = 18000
 

	
 
celeryd.concurrency = 2
 
#celeryd.log.file = celeryd.log
 
celeryd.log.level = debug
 
celeryd.max.tasks.per.child = 1
 

	
 
@@ -197,100 +202,106 @@ set debug = false
 
##################################
 
###       LOGVIEW CONFIG       ###
 
##################################
 
logview.sqlalchemy = #faa
 
logview.pylons.templating = #bfb
 
logview.pylons.util = #eee
 

	
 
#########################################################
 
### DB CONFIGS - EACH DB WILL HAVE IT'S OWN CONFIG    ###
 
#########################################################
 

	
 
# SQLITE [default]
 
sqlalchemy.db1.url = sqlite:///%(here)s/rhodecode.db
 
 
 
# POSTGRESQL
 
# sqlalchemy.db1.url = postgresql://user:pass@localhost/rhodecode
 

	
 
# MySQL
 
# sqlalchemy.db1.url = mysql://user:pass@localhost/rhodecode
 

	
 
# see sqlalchemy docs for others
 

	
 
sqlalchemy.db1.echo = false
 
sqlalchemy.db1.pool_recycle = 3600
 
sqlalchemy.convert_unicode = true
 
sqlalchemy.db1.convert_unicode = true
 

	
 
################################
 
### LOGGING CONFIGURATION   ####
 
################################
 
[loggers]
 
keys = root, routes, rhodecode, sqlalchemy, beaker, templates
 
keys = root, routes, rhodecode, sqlalchemy, beaker, templates, whoosh_indexer
 

	
 
[handlers]
 
keys = console, console_sql
 

	
 
[formatters]
 
keys = generic, color_formatter, color_formatter_sql
 

	
 
#############
 
## LOGGERS ##
 
#############
 
[logger_root]
 
level = NOTSET
 
handlers = console
 

	
 
[logger_routes]
 
level = DEBUG
 
handlers = 
 
qualname = routes.middleware
 
# "level = DEBUG" logs the route matched and routing variables.
 
propagate = 1
 

	
 
[logger_beaker]
 
level = DEBUG
 
handlers = 
 
qualname = beaker.container
 
propagate = 1
 

	
 
[logger_templates]
 
level = INFO
 
handlers = 
 
qualname = pylons.templating
 
propagate = 1
 

	
 
[logger_rhodecode]
 
level = DEBUG
 
handlers = 
 
qualname = rhodecode
 
propagate = 1
 

	
 
[logger_sqlalchemy]
 
level = INFO
 
handlers = console_sql
 
qualname = sqlalchemy.engine
 
propagate = 0
 

	
 
[logger_whoosh_indexer]
 
level = DEBUG
 
handlers = 
 
qualname = whoosh_indexer
 
propagate = 1
 

	
 
##############
 
## HANDLERS ##
 
##############
 

	
 
[handler_console]
 
class = StreamHandler
 
args = (sys.stderr,)
 
level = INFO
 
formatter = generic
 

	
 
[handler_console_sql]
 
class = StreamHandler
 
args = (sys.stderr,)
 
level = WARN
 
formatter = generic
 

	
 
################
 
## FORMATTERS ##
 
################
 

	
 
[formatter_generic]
 
format = %(asctime)s.%(msecs)03d %(levelname)-5.5s [%(name)s] %(message)s
 
datefmt = %Y-%m-%d %H:%M:%S
 

	
rhodecode/config/environment.py
Show inline comments
 
"""Pylons environment configuration"""
 

	
 
import os
 
import logging
 
import rhodecode
 

	
 
from mako.lookup import TemplateLookup
 
from pylons.configuration import PylonsConfig
 
from pylons.error import handle_mako_error
 

	
 
import rhodecode
 
# don't remove this import it does magic for celery
 
from rhodecode.lib import celerypylons
 

	
 
import rhodecode.lib.app_globals as app_globals
 
import rhodecode.lib.helpers
 

	
 
from rhodecode.config.routing import make_map
 
# don't remove this import it does magic for celery
 
from rhodecode.lib import celerypylons, str2bool
 
from rhodecode.lib import engine_from_config
 

	
 
from rhodecode.lib import helpers
 
from rhodecode.lib.auth import set_available_permissions
 
from rhodecode.lib.utils import repo2db_mapper, make_ui, set_rhodecode_config
 
from rhodecode.lib.utils import repo2db_mapper, make_ui, set_rhodecode_config,\
 
    load_rcextensions
 
from rhodecode.lib.utils2 import engine_from_config, str2bool
 
from rhodecode.model import init_model
 
from rhodecode.model.scm import ScmModel
 

	
 
log = logging.getLogger(__name__)
 

	
 

	
 
def load_environment(global_conf, app_conf, initial=False):
 
    """Configure the Pylons environment via the ``pylons.config``
 
    """
 
    Configure the Pylons environment via the ``pylons.config``
 
    object
 
    """
 
    config = PylonsConfig()
 

	
 
    # Pylons paths
 
    root = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
 
    paths = dict(root=root,
 
                 controllers=os.path.join(root, 'controllers'),
 
                 static_files=os.path.join(root, 'public'),
 
                 templates=[os.path.join(root, 'templates')])
 
    paths = dict(
 
        root=root,
 
        controllers=os.path.join(root, 'controllers'),
 
        static_files=os.path.join(root, 'public'),
 
        templates=[os.path.join(root, 'templates')]
 
    )
 

	
 
    # Initialize config with the basic options
 
    config.init_app(global_conf, app_conf, package='rhodecode', paths=paths)
 

	
 
    # store some globals into rhodecode
 
    rhodecode.CELERY_ON = str2bool(config['app_conf'].get('use_celery'))
 

	
 
    config['routes.map'] = make_map(config)
 
    config['pylons.app_globals'] = app_globals.Globals(config)
 
    config['pylons.h'] = rhodecode.lib.helpers
 
    config['pylons.h'] = helpers
 
    rhodecode.CONFIG = config
 

	
 
    load_rcextensions(root_path=config['here'])
 

	
 
    # Setup cache object as early as possible
 
    import pylons
 
    pylons.cache._push_object(config['pylons.app_globals'].cache)
 

	
 
    # Create the Mako TemplateLookup, with the default auto-escaping
 
    config['pylons.app_globals'].mako_lookup = TemplateLookup(
 
        directories=paths['templates'],
 
        error_handler=handle_mako_error,
 
        module_directory=os.path.join(app_conf['cache_dir'], 'templates'),
 
        input_encoding='utf-8', default_filters=['escape'],
 
        imports=['from webhelpers.html import escape'])
 

	
 
    # sets the c attribute access when don't existing attribute are accessed
 
    config['pylons.strict_tmpl_context'] = True
 
    test = os.path.split(config['__file__'])[-1] == 'test.ini'
 
    if test:
 
        from rhodecode.lib.utils import create_test_env, create_test_index
 
        from rhodecode.tests import  TESTS_TMP_PATH
 
        create_test_env(TESTS_TMP_PATH, config)
 
        create_test_index(TESTS_TMP_PATH, config, True)
 

	
 
    # MULTIPLE DB configs
 
    # Setup the SQLAlchemy database engine
 
    sa_engine_db1 = engine_from_config(config, 'sqlalchemy.db1.')
rhodecode/config/rcextensions/__init__.py
Show inline comments
 
new file 100644
 
# Additional mappings that are not present in the pygments lexers
 
# used for building stats
 
# format is {'ext':'Name'} eg. {'py':'Python'}
 
# NOTE: that this will overide any mappings in LANGUAGES_EXTENSIONS_MAP
 
# build by pygments
 
EXTRA_MAPPINGS = {}
 

	
 
#==============================================================================
 
# WHOOSH INDEX EXTENSIONS
 
#==============================================================================
 
# if INDEX_EXTENSIONS is [] it'll use pygments lexers extensions by default.
 
# To set your own just add to this list extensions to index with content
 
INDEX_EXTENSIONS = []
 

	
 
# additional extensions for indexing besides the default from pygments
 
# those get's added to INDEX_EXTENSIONS
 
EXTRA_INDEX_EXTENSIONS = []
 

	
 

	
 
#==============================================================================
 
# POST CREATE REPOSITORY HOOK
 
#==============================================================================
 
# this function will be executed after each repository is created
 
def _crhook(*args, **kwargs):
 
    """
 
    Post create repository HOOK
 
    kwargs available:
 
     :param repo_name:
 
     :param repo_type:
 
     :param description:
 
     :param private:
 
     :param created_on:
 
     :param enable_downloads:
 
     :param repo_id:
 
     :param user_id:
 
     :param enable_statistics:
 
     :param clone_uri:
 
     :param fork_id:
 
     :param group_id:
 
     :param created_by:
 
    """
 
    return 0
 
CREATE_REPO_HOOK = _crhook
 

	
 

	
 
#==============================================================================
 
# POST PUSH HOOK
 
#==============================================================================
 

	
 
# this function will be executed after each push it's runned after the build-in
 
# hook that rhodecode uses for logging pushes
 
def _pushhook(*args, **kwargs):
 
    """
 
    Post push hook
 
    kwargs available:
 

	
 
      :param username: name of user who pushed
 
      :param ip: ip of who pushed
 
      :param action: pull
 
      :param repository: repository name
 
      :param pushed_revs: generator of pushed revisions
 
    """
 
    return 0
 
PUSH_HOOK = _pushhook
 

	
 

	
 
#==============================================================================
 
# POST PULL HOOK
 
#==============================================================================
 

	
 
# this function will be executed after each push it's runned after the build-in
 
# hook that rhodecode uses for logging pushes
 
def _pullhook(*args, **kwargs):
 
    """
 
    Post pull hook
 
    kwargs available::
 

	
 
      :param username: name of user who pulled
 
      :param ip: ip of who pushed
 
      :param action: pull
 
      :param repository: repository name
 
    """
 
    return 0
 
PULL_HOOK = _pullhook
rhodecode/config/rcextensions/make_rcextensions.py
Show inline comments
 
new file 100644
 
# -*- coding: utf-8 -*-
 
"""
 
    rhodecode.config.rcextensions.make_rcextensions
 
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
    Whoosh indexing module for RhodeCode
 

	
 
    :created_on: Mar 6, 2012
 
    :author: marcink
 
    :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>
 
    :license: GPLv3, see COPYING for more details.
 
"""
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
import os
 
import sys
 
import pkg_resources
 
import traceback
 
import logging
 
from os.path import dirname as dn, join as jn
 

	
 
#to get the rhodecode import
 
sys.path.append(dn(dn(dn(os.path.realpath(__file__)))))
 

	
 
from rhodecode.lib.utils import BasePasterCommand, Command, ask_ok
 

	
 
log = logging.getLogger(__name__)
 

	
 

	
 
class MakeRcExt(BasePasterCommand):
 

	
 
    max_args = 1
 
    min_args = 1
 

	
 
    usage = "CONFIG_FILE"
 
    summary = "Creates additional extensions for rhodecode"
 
    group_name = "RhodeCode"
 
    takes_config_file = -1
 
    parser = Command.standard_parser(verbose=True)
 

	
 
    def command(self):
 
        logging.config.fileConfig(self.path_to_ini_file)
 
        from pylons import config
 

	
 
        def _make_file(ext_file):
 
            bdir = os.path.split(ext_file)[0]
 
            if not os.path.isdir(bdir):
 
                os.makedirs(bdir)
 
            with open(ext_file, 'wb') as f:
 
                f.write(tmpl)
 
                log.info('Writen new extensions file to %s' % ext_file)
 

	
 
        here = config['here']
 
        tmpl = pkg_resources.resource_string(
 
            'rhodecode', jn('config', 'rcextensions', '__init__.py')
 
        )
 
        ext_file = jn(here, 'rcextensions', '__init__.py')
 
        if os.path.exists(ext_file):
 
            msg = ('Extension file already exists, do you want '
 
                   'to overwrite it ? [y/n]')
 
            if ask_ok(msg):
 
                _make_file(ext_file)
 
            else:
 
                log.info('nothing done...')
 
        else:
 
            _make_file(ext_file)
 

	
 
    def update_parser(self):
 
        pass
rhodecode/controllers/admin/repos.py
Show inline comments
 
@@ -263,49 +263,48 @@ class ReposController(BaseController):
 
                log.error(traceback.format_exc())
 
                h.flash(_('Cannot delete %s it still contains attached '
 
                          'forks') % repo_name,
 
                        category='warning')
 
            else:
 
                log.error(traceback.format_exc())
 
                h.flash(_('An error occurred during '
 
                          'deletion of %s') % repo_name,
 
                        category='error')
 

	
 
        except Exception, e:
 
            log.error(traceback.format_exc())
 
            h.flash(_('An error occurred during deletion of %s') % repo_name,
 
                    category='error')
 

	
 
        return redirect(url('repos'))
 

	
 
    @HasRepoPermissionAllDecorator('repository.admin')
 
    def delete_perm_user(self, repo_name):
 
        """
 
        DELETE an existing repository permission user
 

	
 
        :param repo_name:
 
        """
 

	
 
        try:
 
            RepoModel().revoke_user_permission(repo=repo_name,
 
                                               user=request.POST['user_id'])
 
            Session.commit()
 
        except Exception:
 
            log.error(traceback.format_exc())
 
            h.flash(_('An error occurred during deletion of repository user'),
 
                    category='error')
 
            raise HTTPInternalServerError()
 

	
 
    @HasRepoPermissionAllDecorator('repository.admin')
 
    def delete_perm_users_group(self, repo_name):
 
        """
 
        DELETE an existing repository permission users group
 

	
 
        :param repo_name:
 
        """
 

	
 
        try:
 
            RepoModel().revoke_users_group_permission(
 
                repo=repo_name, group_name=request.POST['users_group_id']
 
            )
 
            Session.commit()
 
        except Exception:
rhodecode/controllers/admin/users.py
Show inline comments
 
@@ -124,53 +124,54 @@ class UsersController(BaseController):
 
            return htmlfill.render(
 
                render('admin/users/user_edit.html'),
 
                defaults=errors.value,
 
                errors=e,
 
                prefix_error=False,
 
                encoding="UTF-8")
 
        except Exception:
 
            log.error(traceback.format_exc())
 
            h.flash(_('error occurred during update of user %s') \
 
                    % form_result.get('username'), category='error')
 

	
 
        return redirect(url('users'))
 

	
 
    def delete(self, id):
 
        """DELETE /users/id: Delete an existing item"""
 
        # Forms posted to this method should contain a hidden field:
 
        #    <input type="hidden" name="_method" value="DELETE" />
 
        # Or using helpers:
 
        #    h.form(url('delete_user', id=ID),
 
        #           method='delete')
 
        # url('user', id=ID)
 
        user_model = UserModel()
 
        try:
 
            user_model.delete(id)
 
            Session.commit()
 
            h.flash(_('successfully deleted user'), category='success')
 
            Session.commit()
 
        except (UserOwnsReposException, DefaultUserException), e:
 
            h.flash(str(e), category='warning')
 
            h.flash(e, category='warning')
 
        except Exception:
 
            log.error(traceback.format_exc())
 
            h.flash(_('An error occurred during deletion of user'),
 
                    category='error')
 
        return redirect(url('users'))
 

	
 
    def show(self, id, format='html'):
 
        """GET /users/id: Show a specific item"""
 
        # url('user', id=ID)
 

	
 
    def edit(self, id, format='html'):
 
        """GET /users/id/edit: Form to edit an existing item"""
 
        # url('edit_user', id=ID)
 
        c.user = User.get(id)
 
        if not c.user:
 
            return redirect(url('users'))
 
        if c.user.username == 'default':
 
            h.flash(_("You can't edit this user"), category='warning')
 
            return redirect(url('users'))
 
        c.user.permissions = {}
 
        c.granted_permissions = UserModel().fill_perms(c.user)\
 
            .permissions['global']
 

	
 
        defaults = c.user.get_dict()
 
        perm = Permission.get_by_key('hg.create.repository')
 
        defaults.update({'create_repo_perm': UserModel().has_perm(id, perm)})
rhodecode/controllers/admin/users_groups.py
Show inline comments
 
@@ -11,50 +11,51 @@
 
    :license: GPLv3, see COPYING for more details.
 
"""
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

	
 
import logging
 
import traceback
 
import formencode
 

	
 
from formencode import htmlfill
 
from pylons import request, session, tmpl_context as c, url, config
 
from pylons.controllers.util import abort, redirect
 
from pylons.i18n.translation import _
 

	
 
from rhodecode.lib import helpers as h
 
from rhodecode.lib.exceptions import UsersGroupsAssignedException
 
from rhodecode.lib import helpers as h, safe_unicode
 
from rhodecode.lib.utils2 import safe_unicode
 
from rhodecode.lib.auth import LoginRequired, HasPermissionAllDecorator
 
from rhodecode.lib.base import BaseController, render
 

	
 
from rhodecode.model.users_group import UsersGroupModel
 

	
 
from rhodecode.model.db import User, UsersGroup, Permission, UsersGroupToPerm
 
from rhodecode.model.forms import UsersGroupForm
 
from rhodecode.model.meta import Session
 

	
 
log = logging.getLogger(__name__)
 

	
 

	
 
class UsersGroupsController(BaseController):
 
    """REST Controller styled on the Atom Publishing Protocol"""
 
    # To properly map this controller, ensure your config/routing.py
 
    # file has a resource setup:
 
    #     map.resource('users_group', 'users_groups')
 

	
 
    @LoginRequired()
 
    @HasPermissionAllDecorator('hg.admin')
 
    def __before__(self):
 
        c.admin_user = session.get('admin_user')
 
        c.admin_username = session.get('admin_username')
 
        super(UsersGroupsController, self).__before__()
rhodecode/controllers/api/__init__.py
Show inline comments
 
@@ -212,51 +212,51 @@ class JSONRPCController(WSGIController):
 
        """
 
        Implement dispatch interface specified by WSGIController
 
        """
 
        try:
 
            raw_response = self._inspect_call(self._func)
 
            if isinstance(raw_response, HTTPError):
 
                self._error = str(raw_response)
 
        except JSONRPCError, e:
 
            self._error = str(e)
 
        except Exception, e:
 
            log.error('Encountered unhandled exception: %s' \
 
                      % traceback.format_exc())
 
            json_exc = JSONRPCError('Internal server error')
 
            self._error = str(json_exc)
 

	
 
        if self._error is not None:
 
            raw_response = None
 

	
 
        response = dict(id=self._req_id, result=raw_response,
 
                        error=self._error)
 

	
 
        try:
 
            return json.dumps(response)
 
        except TypeError, e:
 
            log.debug('Error encoding response: %s' % e)
 
            log.error('API FAILED. Error encoding response: %s' % e)
 
            return json.dumps(
 
                dict(
 
                    self._req_id,
 
                    id=self._req_id,
 
                    result=None,
 
                    error="Error encoding response"
 
                )
 
            )
 

	
 
    def _find_method(self):
 
        """
 
        Return method named by `self._req_method` in controller if able
 
        """
 
        log.debug('Trying to find JSON-RPC method: %s' % self._req_method)
 
        if self._req_method.startswith('_'):
 
            raise AttributeError("Method not allowed")
 

	
 
        try:
 
            func = getattr(self, self._req_method, None)
 
        except UnicodeEncodeError:
 
            raise AttributeError("Problem decoding unicode in requested "
 
                                 "method name.")
 

	
 
        if isinstance(func, types.MethodType):
 
            return func
 
        else:
 
            raise AttributeError("No such method: %s" % self._req_method)
rhodecode/controllers/api/api.py
Show inline comments
 
@@ -9,58 +9,57 @@
 
    :author: marcink
 
    :copyright: (C) 2011-2012 Marcin Kuzminski <marcin@python-works.com>
 
    :license: GPLv3, see COPYING for more details.
 
"""
 
# This program is free software; you can redistribute it and/or
 
# modify it under the terms of the GNU General Public License
 
# as published by the Free Software Foundation; version 2
 
# of the License or (at your opinion) any later version of the license.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program; if not, write to the Free Software
 
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
 
# MA  02110-1301, USA.
 

	
 
import traceback
 
import logging
 

	
 
from rhodecode.controllers.api import JSONRPCController, JSONRPCError
 
from rhodecode.lib.auth import HasPermissionAllDecorator, \
 
    HasPermissionAnyDecorator, PasswordGenerator
 
    HasPermissionAnyDecorator, PasswordGenerator, AuthUser
 

	
 
from rhodecode.model.meta import Session
 
from rhodecode.model.scm import ScmModel
 
from rhodecode.model.db import User, UsersGroup, RepoGroup, Repository
 
from rhodecode.model.db import User, UsersGroup, Repository
 
from rhodecode.model.repo import RepoModel
 
from rhodecode.model.user import UserModel
 
from rhodecode.model.users_group import UsersGroupModel
 
from rhodecode.model.repos_group import ReposGroupModel
 

	
 
from rhodecode.lib.utils import map_groups
 

	
 
log = logging.getLogger(__name__)
 

	
 

	
 
class ApiController(JSONRPCController):
 
    """
 
    API Controller
 

	
 

	
 
    Each method needs to have USER as argument this is then based on given
 
    API_KEY propagated as instance of user object
 

	
 
    Preferably this should be first argument also
 

	
 

	
 
    Each function should also **raise** JSONRPCError for any
 
    errors that happens
 

	
 
    """
 

	
 
    @HasPermissionAllDecorator('hg.admin')
 
    def pull(self, apiuser, repo_name):
 
        """
 
        Dispatch pull action on given repo
 
@@ -79,71 +78,74 @@ class ApiController(JSONRPCController):
 
        except Exception:
 
            raise JSONRPCError('Unable to pull changes from "%s"' % repo_name)
 

	
 
    @HasPermissionAllDecorator('hg.admin')
 
    def get_user(self, apiuser, userid):
 
        """"
 
        Get a user by username
 

	
 
        :param apiuser:
 
        :param username:
 
        """
 

	
 
        user = UserModel().get_user(userid)
 
        if user is None:
 
            return user
 

	
 
        return dict(
 
            id=user.user_id,
 
            username=user.username,
 
            firstname=user.name,
 
            lastname=user.lastname,
 
            email=user.email,
 
            active=user.active,
 
            admin=user.admin,
 
            ldap_dn=user.ldap_dn
 
            ldap_dn=user.ldap_dn,
 
            last_login=user.last_login,
 
            permissions=AuthUser(user_id=user.user_id).permissions
 
        )
 

	
 
    @HasPermissionAllDecorator('hg.admin')
 
    def get_users(self, apiuser):
 
        """"
 
        Get all users
 

	
 
        :param apiuser:
 
        """
 

	
 
        result = []
 
        for user in User.getAll():
 
            result.append(
 
                dict(
 
                    id=user.user_id,
 
                    username=user.username,
 
                    firstname=user.name,
 
                    lastname=user.lastname,
 
                    email=user.email,
 
                    active=user.active,
 
                    admin=user.admin,
 
                    ldap_dn=user.ldap_dn
 
                    ldap_dn=user.ldap_dn,
 
                    last_login=user.last_login,
 
                )
 
            )
 
        return result
 

	
 
    @HasPermissionAllDecorator('hg.admin')
 
    def create_user(self, apiuser, username, email, password, firstname=None,
 
                    lastname=None, active=True, admin=False, ldap_dn=None):
 
        """
 
        Create new user
 

	
 
        :param apiuser:
 
        :param username:
 
        :param password:
 
        :param email:
 
        :param name:
 
        :param lastname:
 
        :param active:
 
        :param admin:
 
        :param ldap_dn:
 
        """
 
        if User.get_by_username(username):
 
            raise JSONRPCError("user %s already exist" % username)
 

	
 
        if User.get_by_email(email, case_insensitive=True):
 
@@ -261,49 +263,49 @@ class ApiController(JSONRPCController):
 
    @HasPermissionAllDecorator('hg.admin')
 
    def create_users_group(self, apiuser, group_name, active=True):
 
        """
 
        Creates an new usergroup
 

	
 
        :param group_name:
 
        :param active:
 
        """
 

	
 
        if self.get_users_group(apiuser, group_name):
 
            raise JSONRPCError("users group %s already exist" % group_name)
 

	
 
        try:
 
            ug = UsersGroupModel().create(name=group_name, active=active)
 
            Session.commit()
 
            return dict(id=ug.users_group_id,
 
                        msg='created new users group %s' % group_name)
 
        except Exception:
 
            log.error(traceback.format_exc())
 
            raise JSONRPCError('failed to create group %s' % group_name)
 

	
 
    @HasPermissionAllDecorator('hg.admin')
 
    def add_user_to_users_group(self, apiuser, group_name, username):
 
        """"
 
        Add a user to a group
 
        Add a user to a users group
 

	
 
        :param apiuser:
 
        :param group_name:
 
        :param username:
 
        """
 

	
 
        try:
 
            users_group = UsersGroup.get_by_group_name(group_name)
 
            if not users_group:
 
                raise JSONRPCError('unknown users group %s' % group_name)
 

	
 
            user = User.get_by_username(username)
 
            if user is None:
 
                raise JSONRPCError('unknown user %s' % username)
 

	
 
            ugm = UsersGroupModel().add_user_to_group(users_group, user)
 
            success = True if ugm != True else False
 
            msg = 'added member %s to users group %s' % (username, group_name)
 
            msg = msg if success else 'User is already in that group'
 
            Session.commit()
 

	
 
            return dict(
 
                id=ugm.users_group_member_id if ugm != True else None,
 
                success=success,
 
@@ -339,66 +341,66 @@ class ApiController(JSONRPCController):
 
            return dict(success=success, msg=msg)
 
        except Exception:
 
            log.error(traceback.format_exc())
 
            raise JSONRPCError('failed to remove user from group')
 

	
 
    @HasPermissionAnyDecorator('hg.admin')
 
    def get_repo(self, apiuser, repoid):
 
        """"
 
        Get repository by name
 

	
 
        :param apiuser:
 
        :param repo_name:
 
        """
 

	
 
        repo = RepoModel().get_repo(repoid)
 
        if repo is None:
 
            raise JSONRPCError('unknown repository %s' % repo)
 

	
 
        members = []
 
        for user in repo.repo_to_perm:
 
            perm = user.permission.permission_name
 
            user = user.user
 
            members.append(
 
                dict(
 
                    type_="user",
 
                    type="user",
 
                    id=user.user_id,
 
                    username=user.username,
 
                    firstname=user.name,
 
                    lastname=user.lastname,
 
                    email=user.email,
 
                    active=user.active,
 
                    admin=user.admin,
 
                    ldap=user.ldap_dn,
 
                    permission=perm
 
                )
 
            )
 
        for users_group in repo.users_group_to_perm:
 
            perm = users_group.permission.permission_name
 
            users_group = users_group.users_group
 
            members.append(
 
                dict(
 
                    type_="users_group",
 
                    type="users_group",
 
                    id=users_group.users_group_id,
 
                    name=users_group.users_group_name,
 
                    active=users_group.users_group_active,
 
                    permission=perm
 
                )
 
            )
 

	
 
        return dict(
 
            id=repo.repo_id,
 
            repo_name=repo.repo_name,
 
            type=repo.repo_type,
 
            description=repo.description,
 
            members=members
 
        )
 

	
 
    @HasPermissionAnyDecorator('hg.admin')
 
    def get_repos(self, apiuser):
 
        """"
 
        Get all repositories
 

	
 
        :param apiuser:
 
        """
 

	
 
        result = []
 
@@ -443,66 +445,61 @@ class ApiController(JSONRPCController):
 

	
 
    @HasPermissionAnyDecorator('hg.admin', 'hg.create.repository')
 
    def create_repo(self, apiuser, repo_name, owner_name, description='',
 
                    repo_type='hg', private=False, clone_uri=None):
 
        """
 
        Create repository, if clone_url is given it makes a remote clone
 

	
 
        :param apiuser:
 
        :param repo_name:
 
        :param owner_name:
 
        :param description:
 
        :param repo_type:
 
        :param private:
 
        :param clone_uri:
 
        """
 

	
 
        try:
 
            owner = User.get_by_username(owner_name)
 
            if owner is None:
 
                raise JSONRPCError('unknown user %s' % owner_name)
 

	
 
            if Repository.get_by_repo_name(repo_name):
 
                raise JSONRPCError("repo %s already exist" % repo_name)
 

	
 
            groups = repo_name.split('/')
 
            groups = repo_name.split(Repository.url_sep())
 
            real_name = groups[-1]
 
            groups = groups[:-1]
 
            parent_id = None
 
            for g in groups:
 
                group = RepoGroup.get_by_group_name(g)
 
                if not group:
 
                    group = ReposGroupModel().create(g, '', parent_id)
 
                parent_id = group.group_id
 
            # create structure of groups
 
            group = map_groups(repo_name)
 

	
 
            repo = RepoModel().create(
 
                dict(
 
                    repo_name=real_name,
 
                    repo_name_full=repo_name,
 
                    description=description,
 
                    private=private,
 
                    repo_type=repo_type,
 
                    repo_group=parent_id,
 
                    repo_group=group.group_id if group else None,
 
                    clone_uri=clone_uri
 
                ),
 
                owner
 
            )
 
            Session.commit()
 

	
 
            return dict(
 
                id=repo.repo_id,
 
                msg="Created new repository %s" % repo.repo_name
 
            )
 

	
 
        except Exception:
 
            log.error(traceback.format_exc())
 
            raise JSONRPCError('failed to create repository %s' % repo_name)
 

	
 
    @HasPermissionAnyDecorator('hg.admin')
 
    def delete_repo(self, apiuser, repo_name):
 
        """
 
        Deletes a given repository
 

	
 
        :param repo_name:
 
        """
 
        if not Repository.get_by_repo_name(repo_name):
 
            raise JSONRPCError("repo %s does not exist" % repo_name)
rhodecode/controllers/branches.py
Show inline comments
 
@@ -10,49 +10,49 @@
 
    :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>
 
    :license: GPLv3, see COPYING for more details.
 
"""
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

	
 
import logging
 

	
 
from pylons import tmpl_context as c
 
import binascii
 

	
 
from rhodecode.lib.auth import LoginRequired, HasRepoPermissionAnyDecorator
 
from rhodecode.lib.base import BaseRepoController, render
 
from rhodecode.lib.compat import OrderedDict
 
from rhodecode.lib import safe_unicode
 
from rhodecode.lib.utils2 import safe_unicode
 
log = logging.getLogger(__name__)
 

	
 

	
 
class BranchesController(BaseRepoController):
 

	
 
    @LoginRequired()
 
    @HasRepoPermissionAnyDecorator('repository.read', 'repository.write',
 
                                   'repository.admin')
 
    def __before__(self):
 
        super(BranchesController, self).__before__()
 

	
 
    def index(self):
 

	
 
        def _branchtags(localrepo):
 
            bt_closed = {}
 
            for bn, heads in localrepo.branchmap().iteritems():
 
                tip = heads[-1]
 
                if 'close' in localrepo.changelog.read(tip)[5]:
 
                    bt_closed[bn] = tip
 
            return bt_closed
 

	
 
        cs_g = c.rhodecode_repo.get_changeset
 

	
 
        c.repo_closed_branches = {}
rhodecode/controllers/changeset.py
Show inline comments
 
@@ -30,160 +30,169 @@ from webob.exc import HTTPForbidden
 

	
 
from pylons import tmpl_context as c, url, request, response
 
from pylons.i18n.translation import _
 
from pylons.controllers.util import redirect
 
from pylons.decorators import jsonify
 

	
 
from rhodecode.lib.vcs.exceptions import RepositoryError, ChangesetError, \
 
    ChangesetDoesNotExistError
 
from rhodecode.lib.vcs.nodes import FileNode
 

	
 
import rhodecode.lib.helpers as h
 
from rhodecode.lib.auth import LoginRequired, HasRepoPermissionAnyDecorator
 
from rhodecode.lib.base import BaseRepoController, render
 
from rhodecode.lib.utils import EmptyChangeset
 
from rhodecode.lib.compat import OrderedDict
 
from rhodecode.lib import diffs
 
from rhodecode.model.db import ChangesetComment
 
from rhodecode.model.comment import ChangesetCommentsModel
 
from rhodecode.model.meta import Session
 
from rhodecode.lib.diffs import wrapped_diff
 

	
 
log = logging.getLogger(__name__)
 

	
 

	
 
def anchor_url(revision, path):
 
def _update_with_GET(params, GET):
 
    for k in ['diff1', 'diff2', 'diff']:
 
        params[k] += GET.getall(k)
 

	
 

	
 
def anchor_url(revision, path, GET):
 
    fid = h.FID(revision, path)
 
    return h.url.current(anchor=fid, **dict(request.GET))
 
    return h.url.current(anchor=fid, **dict(GET))
 

	
 

	
 
def get_ignore_ws(fid, GET):
 
    ig_ws_global = request.GET.get('ignorews')
 
    ig_ws_global = GET.get('ignorews')
 
    ig_ws = filter(lambda k: k.startswith('WS'), GET.getall(fid))
 
    if ig_ws:
 
        try:
 
            return int(ig_ws[0].split(':')[-1])
 
        except:
 
            pass
 
    return ig_ws_global
 

	
 

	
 
def _ignorews_url(fileid=None):
 

	
 
def _ignorews_url(GET, fileid=None):
 
    fileid = str(fileid) if fileid else None
 
    params = defaultdict(list)
 
    _update_with_GET(params, GET)
 
    lbl = _('show white space')
 
    ig_ws = get_ignore_ws(fileid, request.GET)
 
    ln_ctx = get_line_ctx(fileid, request.GET)
 
    ig_ws = get_ignore_ws(fileid, GET)
 
    ln_ctx = get_line_ctx(fileid, GET)
 
    # global option
 
    if fileid is None:
 
        if ig_ws is None:
 
            params['ignorews'] += [1]
 
            lbl = _('ignore white space')
 
        ctx_key = 'context'
 
        ctx_val = ln_ctx
 
    # per file options
 
    else:
 
        if ig_ws is None:
 
            params[fileid] += ['WS:1']
 
            lbl = _('ignore white space')
 

	
 
        ctx_key = fileid
 
        ctx_val = 'C:%s' % ln_ctx
 
    # if we have passed in ln_ctx pass it along to our params
 
    if ln_ctx:
 
        params[ctx_key] += [ctx_val]
 

	
 
    params['anchor'] = fileid
 
    img = h.image(h.url('/images/icons/text_strikethrough.png'), lbl, class_='icon')
 
    return h.link_to(img, h.url.current(**params), title=lbl, class_='tooltip')
 

	
 

	
 
def get_line_ctx(fid, GET):
 
    ln_ctx_global = request.GET.get('context')
 
    ln_ctx_global = GET.get('context')
 
    ln_ctx = filter(lambda k: k.startswith('C'), GET.getall(fid))
 

	
 
    if ln_ctx:
 
        retval = ln_ctx[0].split(':')[-1]
 
    else:
 
        retval = ln_ctx_global
 

	
 
    try:
 
        return int(retval)
 
    except:
 
        return
 

	
 

	
 
def _context_url(fileid=None):
 
def _context_url(GET, fileid=None):
 
    """
 
    Generates url for context lines
 

	
 
    :param fileid:
 
    """
 
    ig_ws = get_ignore_ws(fileid, request.GET)
 
    ln_ctx = (get_line_ctx(fileid, request.GET) or 3) * 2
 

	
 
    fileid = str(fileid) if fileid else None
 
    ig_ws = get_ignore_ws(fileid, GET)
 
    ln_ctx = (get_line_ctx(fileid, GET) or 3) * 2
 

	
 
    params = defaultdict(list)
 
    _update_with_GET(params, GET)
 

	
 
    # global option
 
    if fileid is None:
 
        if ln_ctx > 0:
 
            params['context'] += [ln_ctx]
 

	
 
        if ig_ws:
 
            ig_ws_key = 'ignorews'
 
            ig_ws_val = 1
 

	
 
    # per file option
 
    else:
 
        params[fileid] += ['C:%s' % ln_ctx]
 
        ig_ws_key = fileid
 
        ig_ws_val = 'WS:%s' % 1
 

	
 
    if ig_ws:
 
        params[ig_ws_key] += [ig_ws_val]
 

	
 
    lbl = _('%s line context') % ln_ctx
 

	
 
    params['anchor'] = fileid
 
    img = h.image(h.url('/images/icons/table_add.png'), lbl, class_='icon')
 
    return h.link_to(img, h.url.current(**params), title=lbl, class_='tooltip')
 

	
 

	
 
class ChangesetController(BaseRepoController):
 

	
 
    @LoginRequired()
 
    @HasRepoPermissionAnyDecorator('repository.read', 'repository.write',
 
                                   'repository.admin')
 
    def __before__(self):
 
        super(ChangesetController, self).__before__()
 
        c.affected_files_cut_off = 60
 

	
 
    def index(self, revision):
 

	
 
        c.anchor_url = anchor_url
 
        c.ignorews_url = _ignorews_url
 
        c.context_url = _context_url
 

	
 
        limit_off = request.GET.get('fulldiff')
 
        #get ranges of revisions if preset
 
        rev_range = revision.split('...')[:2]
 
        enable_comments = True
 
        try:
 
            if len(rev_range) == 2:
 
                enable_comments = False
 
                rev_start = rev_range[0]
 
                rev_end = rev_range[1]
 
                rev_ranges = c.rhodecode_repo.get_changesets(start=rev_start,
 
                                                            end=rev_end)
 
            else:
 
                rev_ranges = [c.rhodecode_repo.get_changeset(revision)]
 

	
 
            c.cs_ranges = list(rev_ranges)
 
            if not c.cs_ranges:
 
                raise RepositoryError('Changeset range returned empty result')
 

	
 
        except (RepositoryError, ChangesetDoesNotExistError, Exception), e:
 
            log.error(traceback.format_exc())
 
            h.flash(str(e), category='warning')
 
            return redirect(url('home'))
 

	
 
        c.changes = OrderedDict()
 

	
 
@@ -199,80 +208,80 @@ class ChangesetController(BaseRepoContro
 
        # Iterate over ranges (default changeset view is always one changeset)
 
        for changeset in c.cs_ranges:
 
            c.comments.extend(ChangesetCommentsModel()\
 
                              .get_comments(c.rhodecode_db_repo.repo_id,
 
                                            changeset.raw_id))
 
            inlines = ChangesetCommentsModel()\
 
                        .get_inline_comments(c.rhodecode_db_repo.repo_id,
 
                                             changeset.raw_id)
 
            c.inline_comments.extend(inlines)
 
            c.changes[changeset.raw_id] = []
 
            try:
 
                changeset_parent = changeset.parents[0]
 
            except IndexError:
 
                changeset_parent = None
 

	
 
            #==================================================================
 
            # ADDED FILES
 
            #==================================================================
 
            for node in changeset.added:
 
                fid = h.FID(revision, node.path)
 
                line_context_lcl = get_line_ctx(fid, request.GET)
 
                ign_whitespace_lcl = get_ignore_ws(fid, request.GET)
 
                lim = self.cut_off_limit
 
                if cumulative_diff > self.cut_off_limit:
 
                    lim = -1
 
                    lim = -1 if limit_off is None else None
 
                size, cs1, cs2, diff, st = wrapped_diff(
 
                    filenode_old=None,
 
                    filenode_new=node,
 
                    cut_off_limit=lim,
 
                    ignore_whitespace=ign_whitespace_lcl,
 
                    line_context=line_context_lcl,
 
                    enable_comments=enable_comments
 
                )
 
                cumulative_diff += size
 
                c.lines_added += st[0]
 
                c.lines_deleted += st[1]
 
                c.changes[changeset.raw_id].append(
 
                    ('added', node, diff, cs1, cs2, st)
 
                )
 

	
 
            #==================================================================
 
            # CHANGED FILES
 
            #==================================================================
 
            for node in changeset.changed:
 
                try:
 
                    filenode_old = changeset_parent.get_node(node.path)
 
                except ChangesetError:
 
                    log.warning('Unable to fetch parent node for diff')
 
                    filenode_old = FileNode(node.path, '', EmptyChangeset())
 

	
 
                fid = h.FID(revision, node.path)
 
                line_context_lcl = get_line_ctx(fid, request.GET)
 
                ign_whitespace_lcl = get_ignore_ws(fid, request.GET)
 
                lim = self.cut_off_limit
 
                if cumulative_diff > self.cut_off_limit:
 
                    lim = -1
 
                    lim = -1 if limit_off is None else None
 
                size, cs1, cs2, diff, st = wrapped_diff(
 
                    filenode_old=filenode_old,
 
                    filenode_new=node,
 
                    cut_off_limit=lim,
 
                    ignore_whitespace=ign_whitespace_lcl,
 
                    line_context=line_context_lcl,
 
                    enable_comments=enable_comments
 
                )
 
                cumulative_diff += size
 
                c.lines_added += st[0]
 
                c.lines_deleted += st[1]
 
                c.changes[changeset.raw_id].append(
 
                    ('changed', node, diff, cs1, cs2, st)
 
                )
 
            #==================================================================
 
            # REMOVED FILES
 
            #==================================================================
 
            for node in changeset.removed:
 
                c.changes[changeset.raw_id].append(
 
                    ('removed', node, None, None, None, (0, 0))
 
                )
 

	
 
        # count inline comments
 
        for path, lines in c.inline_comments:
rhodecode/controllers/feed.py
Show inline comments
 
@@ -7,49 +7,49 @@
 

	
 
    :created_on: Apr 23, 2010
 
    :author: marcink
 
    :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>
 
    :license: GPLv3, see COPYING for more details.
 
"""
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

	
 
import logging
 

	
 
from pylons import url, response, tmpl_context as c
 
from pylons.i18n.translation import _
 

	
 
from rhodecode.lib import safe_unicode
 
from rhodecode.lib.utils2 import safe_unicode
 
from rhodecode.lib.auth import LoginRequired, HasRepoPermissionAnyDecorator
 
from rhodecode.lib.base import BaseRepoController
 

	
 
from webhelpers.feedgenerator import Atom1Feed, Rss201rev2Feed
 

	
 
log = logging.getLogger(__name__)
 

	
 

	
 
class FeedController(BaseRepoController):
 

	
 
    @LoginRequired(api_access=True)
 
    @HasRepoPermissionAnyDecorator('repository.read', 'repository.write',
 
                                   'repository.admin')
 
    def __before__(self):
 
        super(FeedController, self).__before__()
 
        #common values for feeds
 
        self.description = _('Changes on %s repository')
 
        self.title = self.title = _('%s %s feed') % (c.rhodecode_name, '%s')
 
        self.language = 'en-us'
 
        self.ttl = "5"
 
        self.feed_nr = 10
 

	
 
    def _get_title(self, cs):
 
        return "R%s:%s - %s" % (
rhodecode/controllers/files.py
Show inline comments
 
@@ -11,66 +11,68 @@
 
    :license: GPLv3, see COPYING for more details.
 
"""
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

	
 
import os
 
import logging
 
import traceback
 

	
 
from pylons import request, response, tmpl_context as c, url
 
from pylons.i18n.translation import _
 
from pylons.controllers.util import redirect
 
from pylons.decorators import jsonify
 

	
 
from rhodecode.lib.vcs.conf import settings
 
from rhodecode.lib.vcs.exceptions import RepositoryError, ChangesetDoesNotExistError, \
 
    EmptyRepositoryError, ImproperArchiveTypeError, VCSError, \
 
    NodeAlreadyExistsError
 
from rhodecode.lib.vcs.nodes import FileNode
 
from rhodecode.lib import diffs
 
from rhodecode.lib import helpers as h
 

	
 
from rhodecode.lib.compat import OrderedDict
 
from rhodecode.lib import convert_line_endings, detect_mode, safe_str
 
from rhodecode.lib.utils2 import convert_line_endings, detect_mode, safe_str
 
from rhodecode.lib.auth import LoginRequired, HasRepoPermissionAnyDecorator
 
from rhodecode.lib.base import BaseRepoController, render
 
from rhodecode.lib.utils import EmptyChangeset
 
from rhodecode.lib import diffs
 
import rhodecode.lib.helpers as h
 
from rhodecode.lib.vcs.conf import settings
 
from rhodecode.lib.vcs.exceptions import RepositoryError, \
 
    ChangesetDoesNotExistError, EmptyRepositoryError, \
 
    ImproperArchiveTypeError, VCSError, NodeAlreadyExistsError
 
from rhodecode.lib.vcs.nodes import FileNode
 

	
 
from rhodecode.model.repo import RepoModel
 
from rhodecode.model.scm import ScmModel
 

	
 
from rhodecode.controllers.changeset import anchor_url, _ignorews_url,\
 
    _context_url, get_line_ctx, get_ignore_ws
 
from rhodecode.lib.diffs import wrapped_diff
 
from rhodecode.model.scm import ScmModel
 

	
 

	
 
log = logging.getLogger(__name__)
 

	
 

	
 
class FilesController(BaseRepoController):
 

	
 
    @LoginRequired()
 
    def __before__(self):
 
        super(FilesController, self).__before__()
 
        c.cut_off_limit = self.cut_off_limit
 

	
 
    def __get_cs_or_redirect(self, rev, repo_name, redirect_after=True):
 
        """
 
        Safe way to get changeset if error occur it redirects to tip with
 
        proper message
 

	
 
        :param rev: revision to fetch
 
        :param repo_name: repo name to redirect after
 
        """
 

	
 
        try:
 
            return c.rhodecode_repo.get_changeset(rev)
 
        except EmptyRepositoryError, e:
 
            if not redirect_after:
 
@@ -426,49 +428,49 @@ class FilesController(BaseRepoController
 
                                      context=line_context)
 
            diff = diffs.DiffProcessor(_diff, format='gitdiff')
 

	
 
            diff_name = '%s_vs_%s.diff' % (diff1, diff2)
 
            response.content_type = 'text/plain'
 
            response.content_disposition = (
 
                'attachment; filename=%s' % diff_name
 
            )
 
            return diff.raw_diff()
 

	
 
        elif c.action == 'raw':
 
            _diff = diffs.get_gitdiff(node1, node2,
 
                                      ignore_whitespace=ignore_whitespace,
 
                                      context=line_context)
 
            diff = diffs.DiffProcessor(_diff, format='gitdiff')
 
            response.content_type = 'text/plain'
 
            return diff.raw_diff()
 

	
 
        else:
 
            fid = h.FID(diff2, node2.path)
 
            line_context_lcl = get_line_ctx(fid, request.GET)
 
            ign_whitespace_lcl = get_ignore_ws(fid, request.GET)
 

	
 
            lim = request.GET.get('fulldiff') or self.cut_off_limit
 
            _, cs1, cs2, diff, st = wrapped_diff(filenode_old=node1,
 
            _, cs1, cs2, diff, st = diffs.wrapped_diff(filenode_old=node1,
 
                                         filenode_new=node2,
 
                                         cut_off_limit=lim,
 
                                         ignore_whitespace=ign_whitespace_lcl,
 
                                         line_context=line_context_lcl,
 
                                         enable_comments=False)
 

	
 
            c.changes = [('', node2, diff, cs1, cs2, st,)]
 

	
 
        return render('files/file_diff.html')
 

	
 
    def _get_node_history(self, cs, f_path):
 
        changesets = cs.get_file_history(f_path)
 
        hist_l = []
 

	
 
        changesets_group = ([], _("Changesets"))
 
        branches_group = ([], _("Branches"))
 
        tags_group = ([], _("Tags"))
 
        _hg = cs.repository.alias == 'hg'
 
        for chs in changesets:
 
            _branch = '(%s)' % chs.branch if _hg else ''
 
            n_desc = 'r%s:%s %s' % (chs.revision, chs.short_id, _branch)
 
            changesets_group[0].append((chs.raw_id, n_desc,))
 

	
 
        hist_l.append(changesets_group)
rhodecode/controllers/home.py
Show inline comments
 
@@ -23,43 +23,43 @@
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

	
 
import logging
 

	
 
from pylons import tmpl_context as c, request
 
from paste.httpexceptions import HTTPBadRequest
 

	
 
from rhodecode.lib.auth import LoginRequired
 
from rhodecode.lib.base import BaseController, render
 
from rhodecode.model.db import Repository
 

	
 
log = logging.getLogger(__name__)
 

	
 

	
 
class HomeController(BaseController):
 

	
 
    @LoginRequired()
 
    def __before__(self):
 
        super(HomeController, self).__before__()
 

	
 
    def index(self):
 
        c.repos_list = self.scm_model.get_repos()
 
        c.groups = self.scm_model.get_repos_groups()
 

	
 
        c.group = None
 
        return render('/index.html')
 

	
 
    def repo_switcher(self):
 
        if request.is_xhr:
 
            all_repos = Repository.query().order_by(Repository.repo_name).all()
 
            c.repos_list = self.scm_model.get_repos(all_repos,
 
                                                    sort_key='name_sort')
 
            return render('/repo_switcher_list.html')
 
        else:
 
            return HTTPBadRequest()
 

	
 
    def branch_tag_switcher(self, repo_name):
 
        if request.is_xhr:
 
            c.rhodecode_db_repo = Repository.get_by_repo_name(c.repo_name)
 
            c.rhodecode_repo = c.rhodecode_db_repo.scm_instance
 
            return render('/switch_to_list.html')
 
        else:
 
            return HTTPBadRequest()
rhodecode/controllers/summary.py
Show inline comments
 
@@ -5,141 +5,145 @@
 

	
 
    Summary controller for Rhodecode
 

	
 
    :created_on: Apr 18, 2010
 
    :author: marcink
 
    :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>
 
    :license: GPLv3, see COPYING for more details.
 
"""
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

	
 
import traceback
 
import calendar
 
import logging
 
import urllib
 
from time import mktime
 
from datetime import timedelta, date
 
from urlparse import urlparse
 
from rhodecode.lib.compat import product
 

	
 
from rhodecode.lib.vcs.exceptions import ChangesetError, EmptyRepositoryError, \
 
    NodeDoesNotExistError
 

	
 
from pylons import tmpl_context as c, request, url, config
 
from pylons.i18n.translation import _
 

	
 
from beaker.cache import cache_region, region_invalidate
 

	
 
from rhodecode.config.conf import ALL_READMES, ALL_EXTS, LANGUAGES_EXTENSIONS_MAP
 
from rhodecode.model.db import Statistics, CacheInvalidation
 
from rhodecode.lib import ALL_READMES, ALL_EXTS
 
from rhodecode.lib.utils2 import safe_unicode
 
from rhodecode.lib.auth import LoginRequired, HasRepoPermissionAnyDecorator
 
from rhodecode.lib.base import BaseRepoController, render
 
from rhodecode.lib.utils import EmptyChangeset
 
from rhodecode.lib.markup_renderer import MarkupRenderer
 
from rhodecode.lib.celerylib import run_task
 
from rhodecode.lib.celerylib.tasks import get_commits_stats, \
 
    LANGUAGES_EXTENSIONS_MAP
 
from rhodecode.lib.celerylib.tasks import get_commits_stats
 
from rhodecode.lib.helpers import RepoPage
 
from rhodecode.lib.compat import json, OrderedDict
 

	
 
log = logging.getLogger(__name__)
 

	
 
README_FILES = [''.join([x[0][0], x[1][0]]) for x in
 
                    sorted(list(product(ALL_READMES, ALL_EXTS)),
 
                           key=lambda y:y[0][1] + y[1][1])]
 

	
 

	
 
class SummaryController(BaseRepoController):
 

	
 
    @LoginRequired()
 
    @HasRepoPermissionAnyDecorator('repository.read', 'repository.write',
 
                                   'repository.admin')
 
    def __before__(self):
 
        super(SummaryController, self).__before__()
 

	
 
    def index(self, repo_name):
 
        c.dbrepo = dbrepo = c.rhodecode_db_repo
 
        c.following = self.scm_model.is_following_repo(repo_name,
 
                                                self.rhodecode_user.user_id)
 

	
 
        def url_generator(**kw):
 
            return url('shortlog_home', repo_name=repo_name, size=10, **kw)
 

	
 
        c.repo_changesets = RepoPage(c.rhodecode_repo, page=1,
 
                                     items_per_page=10, url=url_generator)
 

	
 
        if self.rhodecode_user.username == 'default':
 
            # for default(anonymous) user we don't need to pass credentials
 
            username = ''
 
            password = ''
 
        else:
 
            username = str(self.rhodecode_user.username)
 
            password = '@'
 

	
 
        parsed_url = urlparse(url.current(qualified=True))
 

	
 
        default_clone_uri = '{scheme}://{user}{pass}{netloc}{path}'
 

	
 
        uri_tmpl = config.get('clone_uri', default_clone_uri)
 
        uri_tmpl = uri_tmpl.replace('{', '%(').replace('}', ')s')
 

	
 
        decoded_path = safe_unicode(urllib.unquote(parsed_url.path))
 
        uri_dict = {
 
           'user': username,
 
           'pass': password,
 
           'scheme': parsed_url.scheme,
 
           'netloc': parsed_url.netloc,
 
           'path': parsed_url.path
 
           'path': decoded_path
 
        }
 

	
 
        uri = uri_tmpl % uri_dict
 
        # generate another clone url by id
 
        uri_dict.update({'path': '/_%s' % c.dbrepo.repo_id})
 
        uri_dict.update(
 
         {'path': decoded_path.replace(repo_name, '_%s' % c.dbrepo.repo_id)}
 
        )
 
        uri_id = uri_tmpl % uri_dict
 

	
 
        c.clone_repo_url = uri
 
        c.clone_repo_url_id = uri_id
 
        c.repo_tags = OrderedDict()
 
        for name, hash in c.rhodecode_repo.tags.items()[:10]:
 
        for name, hash_ in c.rhodecode_repo.tags.items()[:10]:
 
            try:
 
                c.repo_tags[name] = c.rhodecode_repo.get_changeset(hash)
 
                c.repo_tags[name] = c.rhodecode_repo.get_changeset(hash_)
 
            except ChangesetError:
 
                c.repo_tags[name] = EmptyChangeset(hash)
 
                c.repo_tags[name] = EmptyChangeset(hash_)
 

	
 
        c.repo_branches = OrderedDict()
 
        for name, hash in c.rhodecode_repo.branches.items()[:10]:
 
        for name, hash_ in c.rhodecode_repo.branches.items()[:10]:
 
            try:
 
                c.repo_branches[name] = c.rhodecode_repo.get_changeset(hash)
 
                c.repo_branches[name] = c.rhodecode_repo.get_changeset(hash_)
 
            except ChangesetError:
 
                c.repo_branches[name] = EmptyChangeset(hash)
 
                c.repo_branches[name] = EmptyChangeset(hash_)
 

	
 
        td = date.today() + timedelta(days=1)
 
        td_1m = td - timedelta(days=calendar.mdays[td.month])
 
        td_1y = td - timedelta(days=365)
 

	
 
        ts_min_m = mktime(td_1m.timetuple())
 
        ts_min_y = mktime(td_1y.timetuple())
 
        ts_max_y = mktime(td.timetuple())
 

	
 
        if dbrepo.enable_statistics:
 
            c.show_stats = True
 
            c.no_data_msg = _('No data loaded yet')
 
            run_task(get_commits_stats, c.dbrepo.repo_name, ts_min_y, ts_max_y)
 
        else:
 
            c.show_stats = False
 
            c.no_data_msg = _('Statistics are disabled for this repository')
 
        c.ts_min = ts_min_m
 
        c.ts_max = ts_max_y
 

	
 
        stats = self.sa.query(Statistics)\
 
            .filter(Statistics.repository == dbrepo)\
 
            .scalar()
 

	
 
        c.stats_percentage = 0
 
@@ -154,80 +158,80 @@ class SummaryController(BaseRepoControll
 
                               "desc": LANGUAGES_EXTENSIONS_MAP.get(x)})
 
                          for x, y in lang_stats_d.items())
 

	
 
            c.trending_languages = json.dumps(
 
                sorted(lang_stats, reverse=True, key=lambda k: k[1])[:10]
 
            )
 
            last_rev = stats.stat_on_revision + 1
 
            c.repo_last_rev = c.rhodecode_repo.count()\
 
                if c.rhodecode_repo.revisions else 0
 
            if last_rev == 0 or c.repo_last_rev == 0:
 
                pass
 
            else:
 
                c.stats_percentage = '%.2f' % ((float((last_rev)) /
 
                                                c.repo_last_rev) * 100)
 
        else:
 
            c.commit_data = json.dumps({})
 
            c.overview_data = json.dumps([[ts_min_y, 0], [ts_max_y, 10]])
 
            c.trending_languages = json.dumps({})
 
            c.no_data = True
 

	
 
        c.enable_downloads = dbrepo.enable_downloads
 
        if c.enable_downloads:
 
            c.download_options = self._get_download_links(c.rhodecode_repo)
 

	
 
        c.readme_data, c.readme_file = self.__get_readme_data(c.rhodecode_repo)
 
        c.readme_data, c.readme_file = self.__get_readme_data(c.rhodecode_db_repo)
 
        return render('summary/summary.html')
 

	
 
    def __get_readme_data(self, repo):
 

	
 
        @cache_region('long_term')
 
        def _get_readme_from_cache(key):
 
            readme_data = None
 
            readme_file = None
 
            log.debug('Fetching readme file')
 
            try:
 
                cs = repo.get_changeset('tip')
 
                renderer = MarkupRenderer()
 
                for f in README_FILES:
 
                    try:
 
                        readme = cs.get_node(f)
 
                        readme_file = f
 
                        readme_data = renderer.render(readme.content, f)
 
                        log.debug('Found readme %s' % readme_file)
 
                        break
 
                    except NodeDoesNotExistError:
 
                        continue
 
            except ChangesetError:
 
                pass
 
            except EmptyRepositoryError:
 
                pass
 
            except Exception:
 
                log.error(traceback.format_exc())
 

	
 
            return readme_data, readme_file
 

	
 
        key = repo.name + '_README'
 
        key = repo.repo_name + '_README'
 
        inv = CacheInvalidation.invalidate(key)
 
        if inv is not None:
 
            region_invalidate(_get_readme_from_cache, None, key)
 
            CacheInvalidation.set_valid(inv.cache_key)
 
        return _get_readme_from_cache(key)
 

	
 
    def _get_download_links(self, repo):
 

	
 
        download_l = []
 

	
 
        branches_group = ([], _("Branches"))
 
        tags_group = ([], _("Tags"))
 

	
 
        for name, chs in c.rhodecode_repo.branches.items():
 
            #chs = chs.split(':')[-1]
 
            branches_group[0].append((chs, name),)
 
        download_l.append(branches_group)
 

	
 
        for name, chs in c.rhodecode_repo.tags.items():
 
            #chs = chs.split(':')[-1]
 
            tags_group[0].append((chs, name),)
 
        download_l.append(tags_group)
 

	
 
        return download_l
rhodecode/lib/__init__.py
Show inline comments
 
# -*- coding: utf-8 -*-
 
"""
 
    rhodecode.lib.__init__
 
    ~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
    Some simple helper functions
 

	
 
    :created_on: Jan 5, 2011
 
    :author: marcink
 
    :copyright: (C) 2011-2012 Marcin Kuzminski <marcin@python-works.com>
 
    :license: GPLv3, see COPYING for more details.
 
"""
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

	
 
import os
 
import re
 
from rhodecode.lib.vcs.utils.lazy import LazyProperty
 

	
 

	
 
def __get_lem():
 
    from pygments import lexers
 
    from string import lower
 
    from collections import defaultdict
 

	
 
    d = defaultdict(lambda: [])
 

	
 
    def __clean(s):
 
        s = s.lstrip('*')
 
        s = s.lstrip('.')
 

	
 
        if s.find('[') != -1:
 
            exts = []
 
            start, stop = s.find('['), s.find(']')
 

	
 
            for suffix in s[start + 1:stop]:
 
                exts.append(s[:s.find('[')] + suffix)
 
            return map(lower, exts)
 
        else:
 
            return map(lower, [s])
 

	
 
    for lx, t in sorted(lexers.LEXERS.items()):
 
        m = map(__clean, t[-2])
 
        if m:
 
            m = reduce(lambda x, y: x + y, m)
 
            for ext in m:
 
                desc = lx.replace('Lexer', '')
 
                d[ext].append(desc)
 

	
 
    return dict(d)
 

	
 
# language map is also used by whoosh indexer, which for those specified
 
# extensions will index it's content
 
LANGUAGES_EXTENSIONS_MAP = __get_lem()
 

	
 
# Additional mappings that are not present in the pygments lexers
 
# NOTE: that this will overide any mappings in LANGUAGES_EXTENSIONS_MAP
 
ADDITIONAL_MAPPINGS = {'xaml': 'XAML'}
 

	
 
LANGUAGES_EXTENSIONS_MAP.update(ADDITIONAL_MAPPINGS)
 

	
 
# list of readme files to search in file tree and display in summary
 
# attached weights defines the search  order lower is first
 
ALL_READMES = [
 
    ('readme', 0), ('README', 0), ('Readme', 0),
 
    ('doc/readme', 1), ('doc/README', 1), ('doc/Readme', 1),
 
    ('Docs/readme', 2), ('Docs/README', 2), ('Docs/Readme', 2),
 
    ('DOCS/readme', 2), ('DOCS/README', 2), ('DOCS/Readme', 2),
 
    ('docs/readme', 2), ('docs/README', 2), ('docs/Readme', 2),
 
]
 

	
 
# extension together with weights to search lower is first
 
RST_EXTS = [
 
    ('', 0), ('.rst', 1), ('.rest', 1),
 
    ('.RST', 2), ('.REST', 2),
 
    ('.txt', 3), ('.TXT', 3)
 
]
 

	
 
MARKDOWN_EXTS = [
 
    ('.md', 1), ('.MD', 1),
 
    ('.mkdn', 2), ('.MKDN', 2),
 
    ('.mdown', 3), ('.MDOWN', 3),
 
    ('.markdown', 4), ('.MARKDOWN', 4)
 
]
 

	
 
PLAIN_EXTS = [('.text', 2), ('.TEXT', 2)]
 

	
 
ALL_EXTS = MARKDOWN_EXTS + RST_EXTS + PLAIN_EXTS
 

	
 

	
 
def str2bool(_str):
 
    """
 
    returs True/False value from given string, it tries to translate the
 
    string into boolean
 

	
 
    :param _str: string value to translate into boolean
 
    :rtype: boolean
 
    :returns: boolean from given string
 
    """
 
    if _str is None:
 
        return False
 
    if _str in (True, False):
 
        return _str
 
    _str = str(_str).strip().lower()
 
    return _str in ('t', 'true', 'y', 'yes', 'on', '1')
 

	
 

	
 
def convert_line_endings(line, mode):
 
    """
 
    Converts a given line  "line end" accordingly to given mode
 

	
 
    Available modes are::
 
        0 - Unix
 
        1 - Mac
 
        2 - DOS
 

	
 
    :param line: given line to convert
 
    :param mode: mode to convert to
 
    :rtype: str
 
    :return: converted line according to mode
 
    """
 
    from string import replace
 

	
 
    if mode == 0:
 
            line = replace(line, '\r\n', '\n')
 
            line = replace(line, '\r', '\n')
 
    elif mode == 1:
 
            line = replace(line, '\r\n', '\r')
 
            line = replace(line, '\n', '\r')
 
    elif mode == 2:
 
            line = re.sub("\r(?!\n)|(?<!\r)\n", "\r\n", line)
 
    return line
 

	
 

	
 
def detect_mode(line, default):
 
    """
 
    Detects line break for given line, if line break couldn't be found
 
    given default value is returned
 

	
 
    :param line: str line
 
    :param default: default
 
    :rtype: int
 
    :return: value of line end on of 0 - Unix, 1 - Mac, 2 - DOS
 
    """
 
    if line.endswith('\r\n'):
 
        return 2
 
    elif line.endswith('\n'):
 
        return 0
 
    elif line.endswith('\r'):
 
        return 1
 
    else:
 
        return default
 

	
 

	
 
def generate_api_key(username, salt=None):
 
    """
 
    Generates unique API key for given username, if salt is not given
 
    it'll be generated from some random string
 

	
 
    :param username: username as string
 
    :param salt: salt to hash generate KEY
 
    :rtype: str
 
    :returns: sha1 hash from username+salt
 
    """
 
    from tempfile import _RandomNameSequence
 
    import hashlib
 

	
 
    if salt is None:
 
        salt = _RandomNameSequence().next()
 

	
 
    return hashlib.sha1(username + salt).hexdigest()
 

	
 

	
 
def safe_unicode(str_, from_encoding=None):
 
    """
 
    safe unicode function. Does few trick to turn str_ into unicode
 

	
 
    In case of UnicodeDecode error we try to return it with encoding detected
 
    by chardet library if it fails fallback to unicode with errors replaced
 

	
 
    :param str_: string to decode
 
    :rtype: unicode
 
    :returns: unicode object
 
    """
 
    if isinstance(str_, unicode):
 
        return str_
 

	
 
    if not from_encoding:
 
        import rhodecode
 
        DEFAULT_ENCODING = rhodecode.CONFIG.get('default_encoding','utf8')
 
        from_encoding = DEFAULT_ENCODING
 

	
 
    try:
 
        return unicode(str_)
 
    except UnicodeDecodeError:
 
        pass
 

	
 
    try:
 
        return unicode(str_, from_encoding)
 
    except UnicodeDecodeError:
 
        pass
 

	
 
    try:
 
        import chardet
 
        encoding = chardet.detect(str_)['encoding']
 
        if encoding is None:
 
            raise Exception()
 
        return str_.decode(encoding)
 
    except (ImportError, UnicodeDecodeError, Exception):
 
        return unicode(str_, from_encoding, 'replace')
 

	
 

	
 
def safe_str(unicode_, to_encoding=None):
 
    """
 
    safe str function. Does few trick to turn unicode_ into string
 

	
 
    In case of UnicodeEncodeError we try to return it with encoding detected
 
    by chardet library if it fails fallback to string with errors replaced
 

	
 
    :param unicode_: unicode to encode
 
    :rtype: str
 
    :returns: str object
 
    """
 

	
 
    # if it's not basestr cast to str
 
    if not isinstance(unicode_, basestring):
 
        return str(unicode_)
 

	
 
    if isinstance(unicode_, str):
 
        return unicode_
 

	
 
    if not to_encoding:
 
        import rhodecode
 
        DEFAULT_ENCODING = rhodecode.CONFIG.get('default_encoding','utf8')
 
        to_encoding = DEFAULT_ENCODING
 

	
 
    try:
 
        return unicode_.encode(to_encoding)
 
    except UnicodeEncodeError:
 
        pass
 

	
 
    try:
 
        import chardet
 
        encoding = chardet.detect(unicode_)['encoding']
 
        print encoding
 
        if encoding is None:
 
            raise UnicodeEncodeError()
 

	
 
        return unicode_.encode(encoding)
 
    except (ImportError, UnicodeEncodeError):
 
        return unicode_.encode(to_encoding, 'replace')
 

	
 
    return safe_str
 

	
 

	
 
def engine_from_config(configuration, prefix='sqlalchemy.', **kwargs):
 
    """
 
    Custom engine_from_config functions that makes sure we use NullPool for
 
    file based sqlite databases. This prevents errors on sqlite. This only
 
    applies to sqlalchemy versions < 0.7.0
 

	
 
    """
 
    import sqlalchemy
 
    from sqlalchemy import engine_from_config as efc
 
    import logging
 

	
 
    if int(sqlalchemy.__version__.split('.')[1]) < 7:
 

	
 
        # This solution should work for sqlalchemy < 0.7.0, and should use
 
        # proxy=TimerProxy() for execution time profiling
 

	
 
        from sqlalchemy.pool import NullPool
 
        url = configuration[prefix + 'url']
 

	
 
        if url.startswith('sqlite'):
 
            kwargs.update({'poolclass': NullPool})
 
        return efc(configuration, prefix, **kwargs)
 
    else:
 
        import time
 
        from sqlalchemy import event
 
        from sqlalchemy.engine import Engine
 

	
 
        log = logging.getLogger('sqlalchemy.engine')
 
        BLACK, RED, GREEN, YELLOW, BLUE, MAGENTA, CYAN, WHITE = xrange(30, 38)
 
        engine = efc(configuration, prefix, **kwargs)
 

	
 
        def color_sql(sql):
 
            COLOR_SEQ = "\033[1;%dm"
 
            COLOR_SQL = YELLOW
 
            normal = '\x1b[0m'
 
            return ''.join([COLOR_SEQ % COLOR_SQL, sql, normal])
 

	
 
        if configuration['debug']:
 
            #attach events only for debug configuration
 

	
 
            def before_cursor_execute(conn, cursor, statement,
 
                                    parameters, context, executemany):
 
                context._query_start_time = time.time()
 
                log.info(color_sql(">>>>> STARTING QUERY >>>>>"))
 

	
 

	
 
            def after_cursor_execute(conn, cursor, statement,
 
                                    parameters, context, executemany):
 
                total = time.time() - context._query_start_time
 
                log.info(color_sql("<<<<< TOTAL TIME: %f <<<<<" % total))
 

	
 
            event.listen(engine, "before_cursor_execute",
 
                         before_cursor_execute)
 
            event.listen(engine, "after_cursor_execute",
 
                         after_cursor_execute)
 

	
 
    return engine
 

	
 

	
 
def age(curdate):
 
    """
 
    turns a datetime into an age string.
 

	
 
    :param curdate: datetime object
 
    :rtype: unicode
 
    :returns: unicode words describing age
 
    """
 

	
 
    from datetime import datetime
 
    from webhelpers.date import time_ago_in_words
 

	
 
    _ = lambda s: s
 

	
 
    if not curdate:
 
        return ''
 

	
 
    agescales = [(_(u"year"), 3600 * 24 * 365),
 
                 (_(u"month"), 3600 * 24 * 30),
 
                 (_(u"day"), 3600 * 24),
 
                 (_(u"hour"), 3600),
 
                 (_(u"minute"), 60),
 
                 (_(u"second"), 1), ]
 

	
 
    age = datetime.now() - curdate
 
    age_seconds = (age.days * agescales[2][1]) + age.seconds
 
    pos = 1
 
    for scale in agescales:
 
        if scale[1] <= age_seconds:
 
            if pos == 6:
 
                pos = 5
 
            return '%s %s' % (time_ago_in_words(curdate,
 
                                                agescales[pos][0]), _('ago'))
 
        pos += 1
 

	
 
    return _(u'just now')
 

	
 

	
 
def uri_filter(uri):
 
    """
 
    Removes user:password from given url string
 

	
 
    :param uri:
 
    :rtype: unicode
 
    :returns: filtered list of strings
 
    """
 
    if not uri:
 
        return ''
 

	
 
    proto = ''
 

	
 
    for pat in ('https://', 'http://'):
 
        if uri.startswith(pat):
 
            uri = uri[len(pat):]
 
            proto = pat
 
            break
 

	
 
    # remove passwords and username
 
    uri = uri[uri.find('@') + 1:]
 

	
 
    # get the port
 
    cred_pos = uri.find(':')
 
    if cred_pos == -1:
 
        host, port = uri, None
 
    else:
 
        host, port = uri[:cred_pos], uri[cred_pos + 1:]
 

	
 
    return filter(None, [proto, host, port])
 

	
 

	
 
def credentials_filter(uri):
 
    """
 
    Returns a url with removed credentials
 

	
 
    :param uri:
 
    """
 

	
 
    uri = uri_filter(uri)
 
    #check if we have port
 
    if len(uri) > 2 and uri[2]:
 
        uri[2] = ':' + uri[2]
 

	
 
    return ''.join(uri)
 

	
 

	
 
def get_changeset_safe(repo, rev):
 
    """
 
    Safe version of get_changeset if this changeset doesn't exists for a
 
    repo it returns a Dummy one instead
 

	
 
    :param repo:
 
    :param rev:
 
    """
 
    from rhodecode.lib.vcs.backends.base import BaseRepository
 
    from rhodecode.lib.vcs.exceptions import RepositoryError
 
    if not isinstance(repo, BaseRepository):
 
        raise Exception('You must pass an Repository '
 
                        'object as first argument got %s', type(repo))
 

	
 
    try:
 
        cs = repo.get_changeset(rev)
 
    except RepositoryError:
 
        from rhodecode.lib.utils import EmptyChangeset
 
        cs = EmptyChangeset(requested_revision=rev)
 
    return cs
 

	
 

	
 
def get_current_revision(quiet=False):
 
    """
 
    Returns tuple of (number, id) from repository containing this package
 
    or None if repository could not be found.
 

	
 
    :param quiet: prints error for fetching revision if True
 
    """
 

	
 
    try:
 
        from rhodecode.lib.vcs import get_repo
 
        from rhodecode.lib.vcs.utils.helpers import get_scm
 
        repopath = os.path.join(os.path.dirname(__file__), '..', '..')
 
        scm = get_scm(repopath)[0]
 
        repo = get_repo(path=repopath, alias=scm)
 
        tip = repo.get_changeset()
 
        return (tip.revision, tip.short_id)
 
    except Exception, err:
 
        if not quiet:
 
            print ("Cannot retrieve rhodecode's revision. Original error "
 
                   "was: %s" % err)
 
        return None
 

	
 

	
 
def extract_mentioned_users(s):
 
    """
 
    Returns unique usernames from given string s that have @mention
 

	
 
    :param s: string to get mentions
 
    """
 
    usrs = {}
 
    for username in re.findall(r'(?:^@|\s@)(\w+)', s):
 
        usrs[username] = username
 

	
 
    return sorted(usrs.keys())
rhodecode/lib/auth.py
Show inline comments
 
@@ -22,49 +22,49 @@
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

	
 
import random
 
import logging
 
import traceback
 
import hashlib
 

	
 
from tempfile import _RandomNameSequence
 
from decorator import decorator
 

	
 
from pylons import config, url, request
 
from pylons.controllers.util import abort, redirect
 
from pylons.i18n.translation import _
 

	
 
from rhodecode import __platform__, PLATFORM_WIN, PLATFORM_OTHERS
 
from rhodecode.model.meta import Session
 

	
 
if __platform__ in PLATFORM_WIN:
 
    from hashlib import sha256
 
if __platform__ in PLATFORM_OTHERS:
 
    import bcrypt
 

	
 
from rhodecode.lib import str2bool, safe_unicode
 
from rhodecode.lib.utils2 import str2bool, safe_unicode
 
from rhodecode.lib.exceptions import LdapPasswordError, LdapUsernameError
 
from rhodecode.lib.utils import get_repo_slug, get_repos_group_slug
 
from rhodecode.lib.auth_ldap import AuthLdap
 

	
 
from rhodecode.model import meta
 
from rhodecode.model.user import UserModel
 
from rhodecode.model.db import Permission, RhodeCodeSetting, User
 

	
 
log = logging.getLogger(__name__)
 

	
 

	
 
class PasswordGenerator(object):
 
    """
 
    This is a simple class for generating password from different sets of
 
    characters
 
    usage::
 

	
 
        passwd_gen = PasswordGenerator()
 
        #print 8-letter password containing only big and small letters
 
            of alphabet
 
        print passwd_gen.gen_password(8, passwd_gen.ALPHABETS_BIG_SMALL)
 
    """
 
    ALPHABETS_NUM = r'''1234567890'''
 
    ALPHABETS_SMALL = r'''qwertyuiopasdfghjklzxcvbnm'''
 
@@ -500,50 +500,49 @@ class NotAnonymous(object):
 
            return redirect(url('login_home', came_from=p))
 
        else:
 
            return func(*fargs, **fkwargs)
 

	
 

	
 
class PermsDecorator(object):
 
    """Base class for controller decorators"""
 

	
 
    def __init__(self, *required_perms):
 
        available_perms = config['available_permissions']
 
        for perm in required_perms:
 
            if perm not in available_perms:
 
                raise Exception("'%s' permission is not defined" % perm)
 
        self.required_perms = set(required_perms)
 
        self.user_perms = None
 

	
 
    def __call__(self, func):
 
        return decorator(self.__wrapper, func)
 

	
 
    def __wrapper(self, func, *fargs, **fkwargs):
 
        cls = fargs[0]
 
        self.user = cls.rhodecode_user
 
        self.user_perms = self.user.permissions
 
        log.debug('checking %s permissions %s for %s %s',
 
           self.__class__.__name__, self.required_perms, cls,
 
               self.user)
 
           self.__class__.__name__, self.required_perms, cls, self.user)
 

	
 
        if self.check_permissions():
 
            log.debug('Permission granted for %s %s' % (cls, self.user))
 
            return func(*fargs, **fkwargs)
 

	
 
        else:
 
            log.debug('Permission denied for %s %s' % (cls, self.user))
 
            anonymous = self.user.username == 'default'
 

	
 
            if anonymous:
 
                p = url.current()
 

	
 
                import rhodecode.lib.helpers as h
 
                h.flash(_('You need to be a signed in to '
 
                          'view this page'),
 
                        category='warning')
 
                return redirect(url('login_home', came_from=p))
 

	
 
            else:
 
                # redirect with forbidden ret code
 
                return abort(403)
 

	
 
    def check_permissions(self):
 
        """Dummy function for overriding"""
 
@@ -583,48 +582,49 @@ class HasRepoPermissionAllDecorator(Perm
 
    def check_permissions(self):
 
        repo_name = get_repo_slug(request)
 
        try:
 
            user_perms = set([self.user_perms['repositories'][repo_name]])
 
        except KeyError:
 
            return False
 
        if self.required_perms.issubset(user_perms):
 
            return True
 
        return False
 

	
 

	
 
class HasRepoPermissionAnyDecorator(PermsDecorator):
 
    """
 
    Checks for access permission for any of given predicates for specific
 
    repository. In order to fulfill the request any of predicates must be meet
 
    """
 

	
 
    def check_permissions(self):
 
        repo_name = get_repo_slug(request)
 

	
 
        try:
 
            user_perms = set([self.user_perms['repositories'][repo_name]])
 
        except KeyError:
 
            return False
 

	
 
        if self.required_perms.intersection(user_perms):
 
            return True
 
        return False
 

	
 

	
 
class HasReposGroupPermissionAllDecorator(PermsDecorator):
 
    """
 
    Checks for access permission for all given predicates for specific
 
    repository. All of them have to be meet in order to fulfill the request
 
    """
 

	
 
    def check_permissions(self):
 
        group_name = get_repos_group_slug(request)
 
        try:
 
            user_perms = set([self.user_perms['repositories_groups'][group_name]])
 
        except KeyError:
 
            return False
 
        if self.required_perms.issubset(user_perms):
 
            return True
 
        return False
 

	
 

	
 
class HasReposGroupPermissionAnyDecorator(PermsDecorator):
 
    """
 
@@ -634,176 +634,188 @@ class HasReposGroupPermissionAnyDecorato
 

	
 
    def check_permissions(self):
 
        group_name = get_repos_group_slug(request)
 

	
 
        try:
 
            user_perms = set([self.user_perms['repositories_groups'][group_name]])
 
        except KeyError:
 
            return False
 
        if self.required_perms.intersection(user_perms):
 
            return True
 
        return False
 

	
 

	
 
#==============================================================================
 
# CHECK FUNCTIONS
 
#==============================================================================
 
class PermsFunction(object):
 
    """Base function for other check functions"""
 

	
 
    def __init__(self, *perms):
 
        available_perms = config['available_permissions']
 

	
 
        for perm in perms:
 
            if perm not in available_perms:
 
                raise Exception("'%s' permission in not defined" % perm)
 
                raise Exception("'%s' permission is not defined" % perm)
 
        self.required_perms = set(perms)
 
        self.user_perms = None
 
        self.granted_for = ''
 
        self.repo_name = None
 
        self.group_name = None
 

	
 
    def __call__(self, check_Location=''):
 
        user = request.user
 
        log.debug('checking %s %s %s', self.__class__.__name__,
 
                  self.required_perms, user)
 
        cls_name = self.__class__.__name__
 
        check_scope = {
 
            'HasPermissionAll': '',
 
            'HasPermissionAny': '',
 
            'HasRepoPermissionAll': 'repo:%s' % self.repo_name,
 
            'HasRepoPermissionAny': 'repo:%s' % self.repo_name,
 
            'HasReposGroupPermissionAll': 'group:%s' % self.group_name,
 
            'HasReposGroupPermissionAny': 'group:%s' % self.group_name,
 
        }.get(cls_name, '?')
 
        log.debug('checking cls:%s %s usr:%s %s @ %s', cls_name,
 
                  self.required_perms, user, check_scope,
 
                  check_Location or 'unspecified location')
 
        if not user:
 
            log.debug('Empty request user')
 
            return False
 
        self.user_perms = user.permissions
 
        self.granted_for = user
 

	
 
        if self.check_permissions():
 
            log.debug('Permission granted %s @ %s', self.granted_for,
 
            log.debug('Permission granted for user: %s @ %s', user,
 
                      check_Location or 'unspecified location')
 
            return True
 

	
 
        else:
 
            log.debug('Permission denied for %s @ %s', self.granted_for,
 
            log.debug('Permission denied for user: %s @ %s', user,
 
                        check_Location or 'unspecified location')
 
            return False
 

	
 
    def check_permissions(self):
 
        """Dummy function for overriding"""
 
        raise Exception('You have to write this function in child class')
 

	
 

	
 
class HasPermissionAll(PermsFunction):
 
    def check_permissions(self):
 
        if self.required_perms.issubset(self.user_perms.get('global')):
 
            return True
 
        return False
 

	
 

	
 
class HasPermissionAny(PermsFunction):
 
    def check_permissions(self):
 
        if self.required_perms.intersection(self.user_perms.get('global')):
 
            return True
 
        return False
 

	
 

	
 
class HasRepoPermissionAll(PermsFunction):
 

	
 
    def __call__(self, repo_name=None, check_Location=''):
 
        self.repo_name = repo_name
 
        return super(HasRepoPermissionAll, self).__call__(check_Location)
 

	
 
    def check_permissions(self):
 
        if not self.repo_name:
 
            self.repo_name = get_repo_slug(request)
 

	
 
        try:
 
            self.user_perms = set(
 
            self._user_perms = set(
 
                [self.user_perms['repositories'][self.repo_name]]
 
            )
 
        except KeyError:
 
            return False
 
        self.granted_for = self.repo_name
 
        if self.required_perms.issubset(self.user_perms):
 
        if self.required_perms.issubset(self._user_perms):
 
            return True
 
        return False
 

	
 

	
 
class HasRepoPermissionAny(PermsFunction):
 

	
 
    def __call__(self, repo_name=None, check_Location=''):
 
        self.repo_name = repo_name
 
        return super(HasRepoPermissionAny, self).__call__(check_Location)
 

	
 
    def check_permissions(self):
 
        if not self.repo_name:
 
            self.repo_name = get_repo_slug(request)
 

	
 
        try:
 
            self.user_perms = set(
 
            self._user_perms = set(
 
                [self.user_perms['repositories'][self.repo_name]]
 
            )
 
        except KeyError:
 
            return False
 
        self.granted_for = self.repo_name
 
        if self.required_perms.intersection(self.user_perms):
 
        if self.required_perms.intersection(self._user_perms):
 
            return True
 
        return False
 

	
 

	
 
class HasReposGroupPermissionAny(PermsFunction):
 
    def __call__(self, group_name=None, check_Location=''):
 
        self.group_name = group_name
 
        return super(HasReposGroupPermissionAny, self).__call__(check_Location)
 

	
 
    def check_permissions(self):
 
        try:
 
            self.user_perms = set(
 
            self._user_perms = set(
 
                [self.user_perms['repositories_groups'][self.group_name]]
 
            )
 
        except KeyError:
 
            return False
 
        self.granted_for = self.repo_name
 
        if self.required_perms.intersection(self.user_perms):
 
        if self.required_perms.intersection(self._user_perms):
 
            return True
 
        return False
 

	
 

	
 
class HasReposGroupPermissionAll(PermsFunction):
 
    def __call__(self, group_name=None, check_Location=''):
 
        self.group_name = group_name
 
        return super(HasReposGroupPermissionAny, self).__call__(check_Location)
 

	
 
    def check_permissions(self):
 
        try:
 
            self.user_perms = set(
 
            self._user_perms = set(
 
                [self.user_perms['repositories_groups'][self.group_name]]
 
            )
 
        except KeyError:
 
            return False
 
        self.granted_for = self.repo_name
 
        if self.required_perms.issubset(self.user_perms):
 
        if self.required_perms.issubset(self._user_perms):
 
            return True
 
        return False
 

	
 

	
 
#==============================================================================
 
# SPECIAL VERSION TO HANDLE MIDDLEWARE AUTH
 
#==============================================================================
 
class HasPermissionAnyMiddleware(object):
 
    def __init__(self, *perms):
 
        self.required_perms = set(perms)
 

	
 
    def __call__(self, user, repo_name):
 
        # repo_name MUST be unicode, since we handle keys in permission
 
        # dict by unicode
 
        repo_name = safe_unicode(repo_name)
 
        usr = AuthUser(user.user_id)
 
        try:
 
            self.user_perms = set([usr.permissions['repositories'][repo_name]])
 
        except:
 
        except Exception:
 
            log.error('Exception while accessing permissions %s' %
 
                      traceback.format_exc())
 
            self.user_perms = set()
 
        self.granted_for = ''
 
        self.username = user.username
 
        self.repo_name = repo_name
 
        return self.check_permissions()
 

	
 
    def check_permissions(self):
 
        log.debug('checking mercurial protocol '
 
                  'permissions %s for user:%s repository:%s', self.user_perms,
 
                                                self.username, self.repo_name)
 
        if self.required_perms.intersection(self.user_perms):
 
            log.debug('permission granted')
 
            log.debug('permission granted for user:%s on repo:%s' % (
 
                          self.username, self.repo_name
 
                     )
 
            )
 
            return True
 
        log.debug('permission denied')
 
        log.debug('permission denied for user:%s on repo:%s' % (
 
                      self.username, self.repo_name
 
                 )
 
        )
 
        return False
rhodecode/lib/base.py
Show inline comments
 
"""The base Controller API
 

	
 
Provides the BaseController class for subclassing.
 
"""
 
import logging
 
import time
 
import traceback
 

	
 
from paste.auth.basic import AuthBasicAuthenticator
 
from paste.httpexceptions import HTTPUnauthorized, HTTPForbidden
 
from paste.httpheaders import WWW_AUTHENTICATE
 

	
 
from pylons import config, tmpl_context as c, request, session, url
 
from pylons.controllers import WSGIController
 
from pylons.controllers.util import redirect
 
from pylons.templating import render_mako as render
 

	
 
from rhodecode import __version__, BACKENDS
 

	
 
from rhodecode.lib import str2bool, safe_unicode
 
from rhodecode.lib.utils2 import str2bool, safe_unicode
 
from rhodecode.lib.auth import AuthUser, get_container_username, authfunc,\
 
    HasPermissionAnyMiddleware, CookieStoreWrapper
 
from rhodecode.lib.utils import get_repo_slug, invalidate_cache
 
from rhodecode.model import meta
 

	
 
from rhodecode.model.db import Repository
 
from rhodecode.model.notification import NotificationModel
 
from rhodecode.model.scm import ScmModel
 

	
 
log = logging.getLogger(__name__)
 

	
 

	
 
class BasicAuth(AuthBasicAuthenticator):
 

	
 
    def __init__(self, realm, authfunc, auth_http_code=None):
 
        self.realm = realm
 
        self.authfunc = authfunc
 
        self._rc_auth_http_code = auth_http_code
 

	
 
    def build_authentication(self):
 
        head = WWW_AUTHENTICATE.tuples('Basic realm="%s"' % self.realm)
 
        if self._rc_auth_http_code and self._rc_auth_http_code == '403':
 
            # return 403 if alternative http return code is specified in
 
            # RhodeCode config
 
            return HTTPForbidden(headers=head)
 
        return HTTPUnauthorized(headers=head)
 

	
 

	
 
class BaseVCSController(object):
 

	
 
    def __init__(self, application, config):
 
        self.application = application
 
        self.config = config
 
        # base path of repo locations
 
        self.basepath = self.config['base_path']
 
        #authenticate this mercurial request using authfunc
 
        self.authenticate = AuthBasicAuthenticator('', authfunc)
 
        self.authenticate = BasicAuth('', authfunc,
 
                                      config.get('auth_ret_code'))
 
        self.ipaddr = '0.0.0.0'
 

	
 
    def _handle_request(self, environ, start_response):
 
        raise NotImplementedError()
 

	
 
    def _get_by_id(self, repo_name):
 
        """
 
        Get's a special pattern _<ID> from clone url and tries to replace it
 
        with a repository_name for support of _<ID> non changable urls
 

	
 
        :param repo_name:
 
        """
 
        try:
 
            data = repo_name.split('/')
 
            if len(data) >= 2:
 
                by_id = data[1].split('_')
 
                if len(by_id) == 2 and by_id[1].isdigit():
 
                    _repo_name = Repository.get(by_id[1]).repo_name
 
                    data[1] = _repo_name
 
        except:
 
            log.debug('Failed to extract repo_name from id %s' % (
 
                      traceback.format_exc()
 
                      )
 
            )
rhodecode/lib/caching_query.py
Show inline comments
 
@@ -3,49 +3,49 @@
 
Represent persistence structures which allow the usage of
 
Beaker caching with SQLAlchemy.
 

	
 
The three new concepts introduced here are:
 

	
 
 * CachingQuery - a Query subclass that caches and
 
   retrieves results in/from Beaker.
 
 * FromCache - a query option that establishes caching
 
   parameters on a Query
 
 * RelationshipCache - a variant of FromCache which is specific
 
   to a query invoked during a lazy load.
 
 * _params_from_query - extracts value parameters from
 
   a Query.
 

	
 
The rest of what's here are standard SQLAlchemy and
 
Beaker constructs.
 

	
 
"""
 
import beaker
 
from beaker.exceptions import BeakerException
 

	
 
from sqlalchemy.orm.interfaces import MapperOption
 
from sqlalchemy.orm.query import Query
 
from sqlalchemy.sql import visitors
 
from rhodecode.lib import safe_str
 
from rhodecode.lib.utils2 import safe_str
 

	
 

	
 
class CachingQuery(Query):
 
    """A Query subclass which optionally loads full results from a Beaker
 
    cache region.
 

	
 
    The CachingQuery stores additional state that allows it to consult
 
    a Beaker cache before accessing the database:
 

	
 
    * A "region", which is a cache region argument passed to a
 
      Beaker CacheManager, specifies a particular cache configuration
 
      (including backend implementation, expiration times, etc.)
 
    * A "namespace", which is a qualifying name that identifies a
 
      group of keys within the cache.  A query that filters on a name
 
      might use the name "by_name", a query that filters on a date range
 
      to a joined table might use the name "related_date_range".
 

	
 
    When the above state is present, a Beaker cache is retrieved.
 

	
 
    The "namespace" name is first concatenated with
 
    a string composed of the individual entities and columns the Query
 
    requests, i.e. such as ``Query(User.id, User.name)``.
 

	
 
    The Beaker cache is then loaded from the cache manager based
rhodecode/lib/celerylib/__init__.py
Show inline comments
 
@@ -15,49 +15,49 @@
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

	
 
import os
 
import sys
 
import socket
 
import traceback
 
import logging
 
from os.path import dirname as dn, join as jn
 
from pylons import config
 

	
 
from hashlib import md5
 
from decorator import decorator
 

	
 
from rhodecode.lib.vcs.utils.lazy import LazyProperty
 
from rhodecode import CELERY_ON
 
from rhodecode.lib import str2bool, safe_str
 
from rhodecode.lib.utils2 import str2bool, safe_str
 
from rhodecode.lib.pidlock import DaemonLock, LockHeld
 
from rhodecode.model import init_model
 
from rhodecode.model import meta
 
from rhodecode.model.db import Statistics, Repository, User
 

	
 
from sqlalchemy import engine_from_config
 

	
 
from celery.messaging import establish_connection
 

	
 
log = logging.getLogger(__name__)
 

	
 

	
 
class ResultWrapper(object):
 
    def __init__(self, task):
 
        self.task = task
 

	
 
    @LazyProperty
 
    def result(self):
 
        return self.task
 

	
 

	
 
def run_task(task, *args, **kwargs):
 
    if CELERY_ON:
 
        try:
rhodecode/lib/celerylib/tasks.py
Show inline comments
 
@@ -19,49 +19,49 @@
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
from celery.decorators import task
 

	
 
import os
 
import traceback
 
import logging
 
from os.path import join as jn
 

	
 
from time import mktime
 
from operator import itemgetter
 
from string import lower
 

	
 
from pylons import config, url
 
from pylons.i18n.translation import _
 

	
 
from rhodecode.lib.vcs import get_backend
 

	
 
from rhodecode import CELERY_ON
 
from rhodecode.lib import LANGUAGES_EXTENSIONS_MAP, safe_str
 
from rhodecode.lib.utils2 import safe_str
 
from rhodecode.lib.celerylib import run_task, locked_task, dbsession, \
 
    str2bool, __get_lockkey, LockHeld, DaemonLock, get_session
 
from rhodecode.lib.helpers import person
 
from rhodecode.lib.rcmail.smtp_mailer import SmtpMailer
 
from rhodecode.lib.utils import add_cache, action_logger
 
from rhodecode.lib.compat import json, OrderedDict
 

	
 
from rhodecode.model.db import Statistics, Repository, User
 

	
 

	
 
add_cache(config)
 

	
 
__all__ = ['whoosh_index', 'get_commits_stats',
 
           'reset_user_password', 'send_email']
 

	
 

	
 
def get_logger(cls):
 
    if CELERY_ON:
 
        try:
 
            log = cls.get_logger()
 
        except:
 
            log = logging.getLogger(__name__)
 
    else:
 
        log = logging.getLogger(__name__)
 
@@ -126,48 +126,49 @@ def get_commits_stats(repo_name, ts_min_
 

	
 
        if cur_stats is not None:
 
            last_rev = cur_stats.stat_on_revision
 

	
 
        if last_rev == repo.get_changeset().revision and repo_size > 1:
 
            # pass silently without any work if we're not on first revision or
 
            # current state of parsing revision(from db marker) is the
 
            # last revision
 
            lock.release()
 
            return True
 

	
 
        if cur_stats:
 
            commits_by_day_aggregate = OrderedDict(json.loads(
 
                                        cur_stats.commit_activity_combined))
 
            co_day_auth_aggr = json.loads(cur_stats.commit_activity)
 

	
 
        log.debug('starting parsing %s' % parse_limit)
 
        lmktime = mktime
 

	
 
        last_rev = last_rev + 1 if last_rev >= 0 else 0
 
        log.debug('Getting revisions from %s to %s' % (
 
             last_rev, last_rev + parse_limit)
 
        )
 
        for cs in repo[last_rev:last_rev + parse_limit]:
 
            log.debug('parsing %s' % cs)
 
            last_cs = cs  # remember last parsed changeset
 
            k = lmktime([cs.date.timetuple()[0], cs.date.timetuple()[1],
 
                          cs.date.timetuple()[2], 0, 0, 0, 0, 0, 0])
 

	
 
            if akc(cs.author) in co_day_auth_aggr:
 
                try:
 
                    l = [timegetter(x) for x in
 
                         co_day_auth_aggr[akc(cs.author)]['data']]
 
                    time_pos = l.index(k)
 
                except ValueError:
 
                    time_pos = False
 

	
 
                if time_pos >= 0 and time_pos is not False:
 

	
 
                    datadict = \
 
                        co_day_auth_aggr[akc(cs.author)]['data'][time_pos]
 

	
 
                    datadict["commits"] += 1
 
                    datadict["added"] += len(cs.added)
 
                    datadict["changed"] += len(cs.changed)
 
                    datadict["removed"] += len(cs.removed)
 

	
 
                else:
 
                    if k >= ts_min_y and k <= ts_max_y or skip_date_limit:
 
@@ -212,52 +213,52 @@ def get_commits_stats(repo_name, ts_min_
 

	
 
        stats = cur_stats if cur_stats else Statistics()
 
        stats.commit_activity = json.dumps(co_day_auth_aggr)
 
        stats.commit_activity_combined = json.dumps(overview_data)
 

	
 
        log.debug('last revison %s' % last_rev)
 
        leftovers = len(repo.revisions[last_rev:])
 
        log.debug('revisions to parse %s' % leftovers)
 

	
 
        if last_rev == 0 or leftovers < parse_limit:
 
            log.debug('getting code trending stats')
 
            stats.languages = json.dumps(__get_codes_stats(repo_name))
 

	
 
        try:
 
            stats.repository = dbrepo
 
            stats.stat_on_revision = last_cs.revision if last_cs else 0
 
            DBS.add(stats)
 
            DBS.commit()
 
        except:
 
            log.error(traceback.format_exc())
 
            DBS.rollback()
 
            lock.release()
 
            return False
 

	
 
        #final release
 
        # final release
 
        lock.release()
 

	
 
        #execute another task if celery is enabled
 
        # execute another task if celery is enabled
 
        if len(repo.revisions) > 1 and CELERY_ON:
 
            run_task(get_commits_stats, repo_name, ts_min_y, ts_max_y)
 
        return True
 
    except LockHeld:
 
        log.info('LockHeld')
 
        return 'Task with key %s already running' % lockkey
 

	
 
@task(ignore_result=True)
 
@dbsession
 
def send_password_link(user_email):
 
    from rhodecode.model.notification import EmailNotificationModel
 

	
 
    log = get_logger(send_password_link)
 
    DBS = get_session()
 

	
 
    try:
 
        user = User.get_by_email(user_email)
 
        if user:
 
            log.debug('password reset user found %s' % user)
 
            link = url('reset_password_confirmation', key=user.api_key,
 
                       qualified=True)
 
            reg_type = EmailNotificationModel.TYPE_PASSWORD_RESET
 
            body = EmailNotificationModel().get_email_tmpl(reg_type,
 
                                                **{'user':user.short_contact,
 
@@ -306,49 +307,49 @@ def reset_user_password(user_email):
 

	
 
    except:
 
        log.error('Failed to update user password')
 
        log.error(traceback.format_exc())
 

	
 
    return True
 

	
 

	
 
@task(ignore_result=True)
 
@dbsession
 
def send_email(recipients, subject, body, html_body=''):
 
    """
 
    Sends an email with defined parameters from the .ini files.
 

	
 
    :param recipients: list of recipients, it this is empty the defined email
 
        address from field 'email_to' is used instead
 
    :param subject: subject of the mail
 
    :param body: body of the mail
 
    :param html_body: html version of body
 
    """
 
    log = get_logger(send_email)
 
    DBS = get_session()
 

	
 
    email_config = config
 
    subject = "%s %s" % (email_config.get('email_prefix'), subject)
 
    subject = "%s %s" % (email_config.get('email_prefix', ''), subject)
 
    if not recipients:
 
        # if recipients are not defined we send to email_config + all admins
 
        admins = [u.email for u in User.query()
 
                  .filter(User.admin == True).all()]
 
        recipients = [email_config.get('email_to')] + admins
 

	
 
    mail_from = email_config.get('app_email_from', 'RhodeCode')
 
    user = email_config.get('smtp_username')
 
    passwd = email_config.get('smtp_password')
 
    mail_server = email_config.get('smtp_server')
 
    mail_port = email_config.get('smtp_port')
 
    tls = str2bool(email_config.get('smtp_use_tls'))
 
    ssl = str2bool(email_config.get('smtp_use_ssl'))
 
    debug = str2bool(config.get('debug'))
 
    smtp_auth = email_config.get('smtp_auth')
 

	
 
    try:
 
        m = SmtpMailer(mail_from, user, passwd, mail_server, smtp_auth,
 
                       mail_port, ssl, tls, debug=debug)
 
        m.send(recipients, subject, body, html_body)
 
    except:
 
        log.error('Mail sending failed')
 
        log.error(traceback.format_exc())
 
        return False
 
@@ -374,41 +375,42 @@ def create_repo_fork(form_data, cur_user
 
    RepoModel(DBS).create(form_data, cur_user, just_db=True, fork=True)
 

	
 
    alias = form_data['repo_type']
 
    org_repo_name = form_data['org_path']
 
    fork_name = form_data['repo_name_full']
 
    update_after_clone = form_data['update_after_clone']
 
    source_repo_path = os.path.join(base_path, org_repo_name)
 
    destination_fork_path = os.path.join(base_path, fork_name)
 

	
 
    log.info('creating fork of %s as %s', source_repo_path,
 
             destination_fork_path)
 
    backend = get_backend(alias)
 
    backend(safe_str(destination_fork_path), create=True,
 
            src_url=safe_str(source_repo_path),
 
            update_after_clone=update_after_clone)
 
    action_logger(cur_user, 'user_forked_repo:%s' % fork_name,
 
                   org_repo_name, '', DBS)
 

	
 
    action_logger(cur_user, 'user_created_fork:%s' % fork_name,
 
                   fork_name, '', DBS)
 
    # finally commit at latest possible stage
 
    DBS.commit()
 

	
 
def __get_codes_stats(repo_name):
 
    from rhodecode.config.conf import  LANGUAGES_EXTENSIONS_MAP
 
    repo = Repository.get_by_repo_name(repo_name).scm_instance
 

	
 
    tip = repo.get_changeset()
 
    code_stats = {}
 

	
 
    def aggregate(cs):
 
        for f in cs[2]:
 
            ext = lower(f.extension)
 
            if ext in LANGUAGES_EXTENSIONS_MAP.keys() and not f.is_binary:
 
                if ext in code_stats:
 
                    code_stats[ext] += 1
 
                else:
 
                    code_stats[ext] = 1
 

	
 
    map(aggregate, tip.walk('/'))
 

	
 
    return code_stats or {}
rhodecode/lib/celerypylons/commands.py
Show inline comments
 
import rhodecode
 
from rhodecode.lib.utils import BasePasterCommand, Command
 
from rhodecode.lib.utils import BasePasterCommand, Command, load_rcextensions
 
from celery.app import app_or_default
 
from celery.bin import camqadm, celerybeat, celeryd, celeryev
 

	
 
from rhodecode.lib import str2bool
 
from rhodecode.lib.utils2 import str2bool
 

	
 
__all__ = ['CeleryDaemonCommand', 'CeleryBeatCommand',
 
           'CAMQPAdminCommand', 'CeleryEventCommand']
 

	
 

	
 
class CeleryCommand(BasePasterCommand):
 
    """Abstract class implements run methods needed for celery
 

	
 
    Starts the celery worker that uses a paste.deploy configuration
 
    file.
 
    """
 

	
 
    def update_parser(self):
 
        """
 
        Abstract method.  Allows for the class's parser to be updated
 
        before the superclass's `run` method is called.  Necessary to
 
        allow options/arguments to be passed through to the underlying
 
        celery command.
 
        """
 

	
 
        cmd = self.celery_command(app_or_default())
 
        for x in cmd.get_options():
 
            self.parser.add_option(x)
 

	
 
    def command(self):
 
        from pylons import config
 
        try:
 
            CELERY_ON = str2bool(config['app_conf'].get('use_celery'))
 
        except KeyError:
 
            CELERY_ON = False
 

	
 
        if CELERY_ON == False:
 
            raise Exception('Please enable celery_on in .ini config '
 
                            'file before running celeryd')
 
        rhodecode.CELERY_ON = CELERY_ON
 
        load_rcextensions(config['here'])
 
        cmd = self.celery_command(app_or_default())
 
        return cmd.run(**vars(self.options))
 

	
 

	
 
class CeleryDaemonCommand(CeleryCommand):
 
    """Start the celery worker
 

	
 
    Starts the celery worker that uses a paste.deploy configuration
 
    file.
 
    """
 
    usage = 'CONFIG_FILE [celeryd options...]'
 
    summary = __doc__.splitlines()[0]
 
    description = "".join(__doc__.splitlines()[2:])
 

	
 
    parser = Command.standard_parser(quiet=True)
 
    celery_command = celeryd.WorkerCommand
 

	
 

	
 
class CeleryBeatCommand(CeleryCommand):
 
    """Start the celery beat server
 

	
 
    Starts the celery beat server using a paste.deploy configuration
 
    file.
 
    """
 
    usage = 'CONFIG_FILE [celerybeat options...]'
 
    summary = __doc__.splitlines()[0]
 
    description = "".join(__doc__.splitlines()[2:])
 

	
 
    parser = Command.standard_parser(quiet=True)
 
    celery_command = celerybeat.BeatCommand
 

	
 

	
 
class CAMQPAdminCommand(CeleryCommand):
 
    """CAMQP Admin
 

	
 
    CAMQP celery admin tool.
 
    """
 
    usage = 'CONFIG_FILE [camqadm options...]'
 
    summary = __doc__.splitlines()[0]
 
    description = "".join(__doc__.splitlines()[2:])
 

	
 
    parser = Command.standard_parser(quiet=True)
 
    celery_command = camqadm.AMQPAdminCommand
 

	
 

	
 
class CeleryEventCommand(CeleryCommand):
 
    """Celery event command.
 

	
 
    Capture celery events.
 
    """
 
    usage = 'CONFIG_FILE [celeryev options...]'
 
    summary = __doc__.splitlines()[0]
 
    description = "".join(__doc__.splitlines()[2:])
 

	
 
    parser = Command.standard_parser(quiet=True)
 
    celery_command = celeryev.EvCommand
rhodecode/lib/compat.py
Show inline comments
 
@@ -4,72 +4,149 @@
 
    ~~~~~~~~~~~~~~~~~~~~
 

	
 
    Python backward compatibility functions and common libs
 

	
 

	
 
    :created_on: Oct 7, 2011
 
    :author: marcink
 
    :copyright: (C) 2010-2010 Marcin Kuzminski <marcin@python-works.com>
 
    :license: GPLv3, see COPYING for more details.
 
"""
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

	
 
import os
 
import datetime
 
import functools
 
import decimal
 
from rhodecode import __platform__, PLATFORM_WIN
 

	
 
#==============================================================================
 
# json
 
#==============================================================================
 

	
 

	
 
def _is_aware(value):
 
    """
 
    Determines if a given datetime.time is aware.
 

	
 
    The logic is described in Python's docs:
 
    http://docs.python.org/library/datetime.html#datetime.tzinfo
 
    """
 
    return (value.tzinfo is not None
 
            and value.tzinfo.utcoffset(value) is not None)
 

	
 

	
 
def _obj_dump(obj):
 
    """
 
    Custom function for dumping objects to JSON, if obj has __json__ attribute
 
    or method defined it will be used for serialization
 

	
 
    :param obj:
 
    """
 

	
 
    if isinstance(obj, complex):
 
        return [obj.real, obj.imag]
 
    # See "Date Time String Format" in the ECMA-262 specification.
 
    # some code borrowed from django 1.4
 
    elif isinstance(obj, datetime.datetime):
 
        r = obj.isoformat()
 
        if obj.microsecond:
 
            r = r[:23] + r[26:]
 
        if r.endswith('+00:00'):
 
            r = r[:-6] + 'Z'
 
        return r
 
    elif isinstance(obj, datetime.date):
 
        return obj.isoformat()
 
    elif isinstance(obj, decimal.Decimal):
 
        return str(obj)
 
    elif isinstance(obj, datetime.time):
 
        if _is_aware(obj):
 
            raise ValueError("JSON can't represent timezone-aware times.")
 
        r = obj.isoformat()
 
        if obj.microsecond:
 
            r = r[:12]
 
        return r
 
    elif isinstance(obj, set):
 
        return list(obj)
 
    elif isinstance(obj, OrderedDict):
 
        return obj.as_dict()
 
    elif hasattr(obj, '__json__'):
 
        if callable(obj.__json__):
 
            return obj.__json__()
 
        else:
 
            return obj.__json__
 
    else:
 
        raise NotImplementedError
 

	
 
try:
 
    import json
 

	
 
    # extended JSON encoder for json
 
    class ExtendedEncoder(json.JSONEncoder):
 
        def default(self, obj):
 
            try:
 
                return _obj_dump(obj)
 
            except NotImplementedError:
 
                pass
 
            return json.JSONEncoder.default(self, obj)
 
    # monkey-patch JSON encoder to use extended version
 
    json.dumps = functools.partial(json.dumps, cls=ExtendedEncoder)
 
except ImportError:
 
    import simplejson as json
 

	
 
    def extended_encode(obj):
 
        try:
 
            return _obj_dump(obj)
 
        except NotImplementedError:
 
            pass
 
        raise TypeError("%r is not JSON serializable" % (obj,))
 
    json.dumps = functools.partial(json.dumps, default=extended_encode)
 

	
 

	
 
#==============================================================================
 
# izip_longest
 
#==============================================================================
 
try:
 
    from itertools import izip_longest
 
except ImportError:
 
    import itertools
 

	
 
    def izip_longest(*args, **kwds): # noqa
 
    def izip_longest(*args, **kwds):
 
        fillvalue = kwds.get("fillvalue")
 

	
 
        def sentinel(counter=([fillvalue] * (len(args) - 1)).pop):
 
            yield counter() # yields the fillvalue, or raises IndexError
 
            yield counter()  # yields the fillvalue, or raises IndexError
 

	
 
        fillers = itertools.repeat(fillvalue)
 
        iters = [itertools.chain(it, sentinel(), fillers)
 
                    for it in args]
 
        try:
 
            for tup in itertools.izip(*iters):
 
                yield tup
 
        except IndexError:
 
            pass
 

	
 

	
 
#==============================================================================
 
# OrderedDict
 
#==============================================================================
 

	
 
# Python Software Foundation License
 

	
 
# XXX: it feels like using the class with "is" and "is not" instead of "==" and
 
# "!=" should be faster.
 
class _Nil(object):
 

	
 
    def __repr__(self):
 
        return "nil"
 

	
rhodecode/lib/db_manage.py
Show inline comments
 
@@ -355,62 +355,66 @@ class DbManage(object):
 
            setting = RhodeCodeSetting(k, v)
 
            self.sa.add(setting)
 

	
 
    def fixup_groups(self):
 
        def_usr = User.get_by_username('default')
 
        for g in RepoGroup.query().all():
 
            g.group_name = g.get_new_name(g.name)
 
            self.sa.add(g)
 
            # get default perm
 
            default = UserRepoGroupToPerm.query()\
 
                .filter(UserRepoGroupToPerm.group == g)\
 
                .filter(UserRepoGroupToPerm.user == def_usr)\
 
                .scalar()
 

	
 
            if default is None:
 
                log.debug('missing default permission for group %s adding' % g)
 
                ReposGroupModel()._create_default_perms(g)
 

	
 
    def config_prompt(self, test_repo_path='', retries=3):
 
        if retries == 3:
 
            log.info('Setting up repositories config')
 

	
 
        if not self.tests and not test_repo_path:
 
            path = raw_input(
 
                 'Enter a valid path to store repositories. '
 
                 'Enter a valid absolute path to store repositories. '
 
                 'All repositories in that path will be added automatically:'
 
            )
 
        else:
 
            path = test_repo_path
 
        path_ok = True
 

	
 
        # check proper dir
 
        if not os.path.isdir(path):
 
            path_ok = False
 
            log.error('Given path %s is not a valid directory' % path)
 

	
 
        elif not os.path.isabs(path):
 
            path_ok = False
 
            log.error('Given path %s is not an absolute path' % path)
 

	
 
        # check write access
 
        if not os.access(path, os.W_OK) and path_ok:
 
        elif not os.access(path, os.W_OK) and path_ok:
 
            path_ok = False
 
            log.error('No write permission to given path %s' % path)
 

	
 
        if retries == 0:
 
            sys.exit('max retries reached')
 
        if path_ok is False:
 
            retries -= 1
 
            return self.config_prompt(test_repo_path, retries)
 

	
 
        return path
 

	
 
    def create_settings(self, path):
 

	
 
        self.create_ui_settings()
 

	
 
        #HG UI OPTIONS
 
        web1 = RhodeCodeUi()
 
        web1.ui_section = 'web'
 
        web1.ui_key = 'push_ssl'
 
        web1.ui_value = 'false'
 

	
 
        web2 = RhodeCodeUi()
 
        web2.ui_section = 'web'
 
        web2.ui_key = 'allow_archive'
rhodecode/lib/dbmigrate/migrate/exceptions.py
Show inline comments
 
@@ -50,38 +50,35 @@ class PathError(Error):
 
class PathNotFoundError(PathError):
 
    """A path with no file was required; found a file."""
 

	
 

	
 
class PathFoundError(PathError):
 
    """A path with a file was required; found no file."""
 

	
 

	
 
class RepositoryError(Error):
 
    """Base class for repository errors."""
 

	
 

	
 
class InvalidRepositoryError(RepositoryError):
 
    """Invalid repository error."""
 

	
 

	
 
class ScriptError(Error):
 
    """Base class for script errors."""
 

	
 

	
 
class InvalidScriptError(ScriptError):
 
    """Invalid script error."""
 

	
 

	
 
class InvalidVersionError(Error):
 
    """Invalid version error."""
 

	
 
# migrate.changeset
 

	
 
class NotSupportedError(Error):
 
    """Not supported error"""
 

	
 

	
 
class InvalidConstraintError(Error):
 
    """Invalid constraint error"""
 

	
 
class MigrateDeprecationWarning(DeprecationWarning):
 
    """Warning for deprecated features in Migrate"""
rhodecode/lib/dbmigrate/schema/db_1_2_0.py
Show inline comments
 
@@ -18,49 +18,49 @@
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

	
 
import os
 
import logging
 
import datetime
 
import traceback
 
from datetime import date
 

	
 
from sqlalchemy import *
 
from sqlalchemy.ext.hybrid import hybrid_property
 
from sqlalchemy.orm import relationship, joinedload, class_mapper, validates
 
from beaker.cache import cache_region, region_invalidate
 

	
 
from rhodecode.lib.vcs import get_backend
 
from rhodecode.lib.vcs.utils.helpers import get_scm
 
from rhodecode.lib.vcs.exceptions import VCSError
 
from rhodecode.lib.vcs.utils.lazy import LazyProperty
 

	
 
from rhodecode.lib import str2bool, safe_str, get_changeset_safe, \
 
from rhodecode.lib.utils2 import str2bool, safe_str, get_changeset_safe, \
 
    generate_api_key, safe_unicode
 
from rhodecode.lib.exceptions import UsersGroupsAssignedException
 
from rhodecode.lib.compat import json
 

	
 
from rhodecode.model.meta import Base, Session
 
from rhodecode.lib.caching_query import FromCache
 

	
 

	
 
log = logging.getLogger(__name__)
 

	
 
#==============================================================================
 
# BASE CLASSES
 
#==============================================================================
 

	
 
class ModelSerializer(json.JSONEncoder):
 
    """
 
    Simple Serializer for JSON,
 

	
 
    usage::
 

	
 
        to make object customized for serialization implement a __json__
 
        method that will return a dict for serialization into json
 

	
 
    example::
 
@@ -696,61 +696,60 @@ class Repository(Base, BaseModel):
 
        try:
 
            alias = get_scm(repo_full_path)[0]
 
            log.debug('Creating instance of %s repository' % alias)
 
            backend = get_backend(alias)
 
        except VCSError:
 
            log.error(traceback.format_exc())
 
            log.error('Perhaps this repository is in db and not in '
 
                      'filesystem run rescan repositories with '
 
                      '"destroy old data " option from admin panel')
 
            return
 

	
 
        if alias == 'hg':
 

	
 
            repo = backend(safe_str(repo_full_path), create=False,
 
                           baseui=self._ui)
 
            # skip hidden web repository
 
            if repo._get_hidden():
 
                return
 
        else:
 
            repo = backend(repo_full_path, create=False)
 

	
 
        return repo
 

	
 

	
 
class RepoGroup(Base, BaseModel):
 
class Group(Base, BaseModel):
 
    __tablename__ = 'groups'
 
    __table_args__ = (UniqueConstraint('group_name', 'group_parent_id'),
 
                      CheckConstraint('group_id != group_parent_id'), {'extend_existing':True},)
 
    __mapper_args__ = {'order_by':'group_name'}
 

	
 
    group_id = Column("group_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
 
    group_name = Column("group_name", String(length=255, convert_unicode=False, assert_unicode=None), nullable=False, unique=True, default=None)
 
    group_parent_id = Column("group_parent_id", Integer(), ForeignKey('groups.group_id'), nullable=True, unique=None, default=None)
 
    group_description = Column("group_description", String(length=10000, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
 

	
 
    parent_group = relationship('RepoGroup', remote_side=group_id)
 

	
 
    parent_group = relationship('Group', remote_side=group_id)
 

	
 
    def __init__(self, group_name='', parent_group=None):
 
        self.group_name = group_name
 
        self.parent_group = parent_group
 

	
 
    def __repr__(self):
 
        return "<%s('%s:%s')>" % (self.__class__.__name__, self.group_id,
 
                                  self.group_name)
 

	
 
    @classmethod
 
    def groups_choices(cls):
 
        from webhelpers.html import literal as _literal
 
        repo_groups = [('', '')]
 
        sep = ' &raquo; '
 
        _name = lambda k: _literal(sep.join(k))
 

	
 
        repo_groups.extend([(x.group_id, _name(x.full_path_splitted))
 
                              for x in cls.query().all()])
 

	
 
        repo_groups = sorted(repo_groups, key=lambda t: t[1].split(sep)[0])
 
        return repo_groups
 

	
 
    @classmethod
 
    def url_sep(cls):
rhodecode/lib/helpers.py
Show inline comments
 
@@ -18,147 +18,160 @@ from pylons.i18n.translation import _, u
 
from hashlib import md5
 

	
 
from webhelpers.html import literal, HTML, escape
 
from webhelpers.html.tools import *
 
from webhelpers.html.builder import make_tag
 
from webhelpers.html.tags import auto_discovery_link, checkbox, css_classes, \
 
    end_form, file, form, hidden, image, javascript_link, link_to, \
 
    link_to_if, link_to_unless, ol, required_legend, select, stylesheet_link, \
 
    submit, text, password, textarea, title, ul, xml_declaration, radio
 
from webhelpers.html.tools import auto_link, button_to, highlight, \
 
    js_obfuscate, mail_to, strip_links, strip_tags, tag_re
 
from webhelpers.number import format_byte_size, format_bit_size
 
from webhelpers.pylonslib import Flash as _Flash
 
from webhelpers.pylonslib.secure_form import secure_form
 
from webhelpers.text import chop_at, collapse, convert_accented_entities, \
 
    convert_misc_entities, lchop, plural, rchop, remove_formatting, \
 
    replace_whitespace, urlify, truncate, wrap_paragraphs
 
from webhelpers.date import time_ago_in_words
 
from webhelpers.paginate import Page
 
from webhelpers.html.tags import _set_input_attrs, _set_id_attr, \
 
    convert_boolean_attrs, NotGiven, _make_safe_id_component
 

	
 
from rhodecode.lib.annotate import annotate_highlight
 
from rhodecode.lib.utils import repo_name_slug
 
from rhodecode.lib import str2bool, safe_unicode, safe_str, get_changeset_safe
 
from rhodecode.lib.utils2 import str2bool, safe_unicode, safe_str, \
 
    get_changeset_safe
 
from rhodecode.lib.markup_renderer import MarkupRenderer
 

	
 
log = logging.getLogger(__name__)
 

	
 

	
 
def shorter(text, size=20):
 
    postfix = '...'
 
    if len(text) > size:
 
        return text[:size - len(postfix)] + postfix
 
    return text
 

	
 

	
 
def _reset(name, value=None, id=NotGiven, type="reset", **attrs):
 
    """
 
    Reset button
 
    """
 
    _set_input_attrs(attrs, type, name, value)
 
    _set_id_attr(attrs, id, name)
 
    convert_boolean_attrs(attrs, ["disabled"])
 
    return HTML.input(**attrs)
 

	
 
reset = _reset
 
safeid = _make_safe_id_component
 

	
 

	
 
def FID(raw_id, path):
 
    """
 
    Creates a uniqe ID for filenode based on it's hash of path and revision
 
    it's safe to use in urls
 

	
 
    :param raw_id:
 
    :param path:
 
    """
 

	
 
    return 'C-%s-%s' % (short_id(raw_id), md5(path).hexdigest()[:12])
 
    return 'C-%s-%s' % (short_id(raw_id), md5(safe_str(path)).hexdigest()[:12])
 

	
 

	
 
def get_token():
 
    """Return the current authentication token, creating one if one doesn't
 
    already exist.
 
    """
 
    token_key = "_authentication_token"
 
    from pylons import session
 
    if not token_key in session:
 
        try:
 
            token = hashlib.sha1(str(random.getrandbits(128))).hexdigest()
 
        except AttributeError: # Python < 2.4
 
            token = hashlib.sha1(str(random.randrange(2 ** 128))).hexdigest()
 
        session[token_key] = token
 
        if hasattr(session, 'save'):
 
            session.save()
 
    return session[token_key]
 

	
 

	
 
class _GetError(object):
 
    """Get error from form_errors, and represent it as span wrapped error
 
    message
 

	
 
    :param field_name: field to fetch errors for
 
    :param form_errors: form errors dict
 
    """
 

	
 
    def __call__(self, field_name, form_errors):
 
        tmpl = """<span class="error_msg">%s</span>"""
 
        if form_errors and form_errors.has_key(field_name):
 
            return literal(tmpl % form_errors.get(field_name))
 

	
 
get_error = _GetError()
 

	
 

	
 
class _ToolTip(object):
 

	
 
    def __call__(self, tooltip_title, trim_at=50):
 
        """Special function just to wrap our text into nice formatted
 
        autowrapped text
 

	
 
        :param tooltip_title:
 
        """
 
        return escape(tooltip_title)
 
tooltip = _ToolTip()
 

	
 

	
 
class _FilesBreadCrumbs(object):
 

	
 
    def __call__(self, repo_name, rev, paths):
 
        if isinstance(paths, str):
 
            paths = safe_unicode(paths)
 
        url_l = [link_to(repo_name, url('files_home',
 
                                        repo_name=repo_name,
 
                                        revision=rev, f_path=''))]
 
        paths_l = paths.split('/')
 
        for cnt, p in enumerate(paths_l):
 
            if p != '':
 
                url_l.append(link_to(p,
 
                                     url('files_home',
 
                                         repo_name=repo_name,
 
                                         revision=rev,
 
                                         f_path='/'.join(paths_l[:cnt + 1])
 
                                         )
 
                                     )
 
                             )
 

	
 
        return literal('/'.join(url_l))
 

	
 
files_breadcrumbs = _FilesBreadCrumbs()
 

	
 

	
 
class CodeHtmlFormatter(HtmlFormatter):
 
    """My code Html Formatter for source codes
 
    """
 
    My code Html Formatter for source codes
 
    """
 

	
 
    def wrap(self, source, outfile):
 
        return self._wrap_div(self._wrap_pre(self._wrap_code(source)))
 

	
 
    def _wrap_code(self, source):
 
        for cnt, it in enumerate(source):
 
            i, t = it
 
            t = '<div id="L%s">%s</div>' % (cnt + 1, t)
 
            yield i, t
 

	
 
    def _wrap_tablelinenos(self, inner):
 
        dummyoutfile = StringIO.StringIO()
 
        lncount = 0
 
        for t, line in inner:
 
            if t:
 
                lncount += 1
 
            dummyoutfile.write(line)
 

	
 
        fl = self.linenostart
 
        mw = len(str(lncount + fl - 1))
 
        sp = self.linenospecial
 
        st = self.linenostep
 
        la = self.lineanchors
 
@@ -298,49 +311,49 @@ def pygmentize_annotation(repo_name, fil
 
                    url('changeset_home', repo_name=repo_name,
 
                        revision=changeset.raw_id),
 
                    style=get_color_string(changeset.raw_id),
 
                    class_='tooltip',
 
                    title=tooltip_html
 
                  )
 

	
 
            uri += '\n'
 
            return uri
 
        return _url_func
 

	
 
    return literal(annotate_highlight(filenode, url_func(repo_name), **kwargs))
 

	
 

	
 
def is_following_repo(repo_name, user_id):
 
    from rhodecode.model.scm import ScmModel
 
    return ScmModel().is_following_repo(repo_name, user_id)
 

	
 
flash = _Flash()
 

	
 
#==============================================================================
 
# SCM FILTERS available via h.
 
#==============================================================================
 
from rhodecode.lib.vcs.utils import author_name, author_email
 
from rhodecode.lib import credentials_filter, age as _age
 
from rhodecode.lib.utils2 import credentials_filter, age as _age
 
from rhodecode.model.db import User
 

	
 
age = lambda  x: _age(x)
 
capitalize = lambda x: x.capitalize()
 
email = author_email
 
short_id = lambda x: x[:12]
 
hide_credentials = lambda x: ''.join(credentials_filter(x))
 

	
 

	
 
def is_git(repository):
 
    if hasattr(repository, 'alias'):
 
        _type = repository.alias
 
    elif hasattr(repository, 'repo_type'):
 
        _type = repository.repo_type
 
    else:
 
        _type = repository
 
    return _type == 'git'
 

	
 

	
 
def is_hg(repository):
 
    if hasattr(repository, 'alias'):
 
        _type = repository.alias
 
    elif hasattr(repository, 'repo_type'):
 
        _type = repository.repo_type
 
@@ -738,141 +751,152 @@ def fancy_file_stats(stats):
 

	
 
    :param stats: two element list of added/deleted lines of code
 
    """
 

	
 
    a, d, t = stats[0], stats[1], stats[0] + stats[1]
 
    width = 100
 
    unit = float(width) / (t or 1)
 

	
 
    # needs > 9% of width to be visible or 0 to be hidden
 
    a_p = max(9, unit * a) if a > 0 else 0
 
    d_p = max(9, unit * d) if d > 0 else 0
 
    p_sum = a_p + d_p
 

	
 
    if p_sum > width:
 
        #adjust the percentage to be == 100% since we adjusted to 9
 
        if a_p > d_p:
 
            a_p = a_p - (p_sum - width)
 
        else:
 
            d_p = d_p - (p_sum - width)
 

	
 
    a_v = a if a > 0 else ''
 
    d_v = d if d > 0 else ''
 

	
 
    def cgen(l_type):
 
        mapping = {'tr': 'top-right-rounded-corner',
 
                   'tl': 'top-left-rounded-corner',
 
                   'br': 'bottom-right-rounded-corner',
 
                   'bl': 'bottom-left-rounded-corner'}
 
        mapping = {'tr': 'top-right-rounded-corner-mid',
 
                   'tl': 'top-left-rounded-corner-mid',
 
                   'br': 'bottom-right-rounded-corner-mid',
 
                   'bl': 'bottom-left-rounded-corner-mid'}
 
        map_getter = lambda x: mapping[x]
 

	
 
        if l_type == 'a' and d_v:
 
            #case when added and deleted are present
 
            return ' '.join(map(map_getter, ['tl', 'bl']))
 

	
 
        if l_type == 'a' and not d_v:
 
            return ' '.join(map(map_getter, ['tr', 'br', 'tl', 'bl']))
 

	
 
        if l_type == 'd' and a_v:
 
            return ' '.join(map(map_getter, ['tr', 'br']))
 

	
 
        if l_type == 'd' and not a_v:
 
            return ' '.join(map(map_getter, ['tr', 'br', 'tl', 'bl']))
 

	
 
    d_a = '<div class="added %s" style="width:%s%%">%s</div>' % (
 
        cgen('a'), a_p, a_v
 
    )
 
    d_d = '<div class="deleted %s" style="width:%s%%">%s</div>' % (
 
        cgen('d'), d_p, d_v
 
    )
 
    return literal('<div style="width:%spx">%s%s</div>' % (width, d_a, d_d))
 

	
 

	
 
def urlify_text(text_):
 
    import re
 

	
 
    url_pat = re.compile(r'''(http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+]'''
 
                         '''|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+)''')
 

	
 
    def url_func(match_obj):
 
        url_full = match_obj.groups()[0]
 
        return '<a href="%(url)s">%(url)s</a>' % ({'url': url_full})
 

	
 
    return literal(url_pat.sub(url_func, text_))
 

	
 

	
 
def urlify_changesets(text_, repository):
 
    """
 
    Extract revision ids from changeset and make link from them
 

	
 
    :param text_:
 
    :param repository:
 
    """
 
    import re
 
    URL_PAT = re.compile(r'([0-9a-fA-F]{12,})')
 

	
 
    def url_func(match_obj):
 
        rev = match_obj.groups()[0]
 
        pref = ''
 
        if match_obj.group().startswith(' '):
 
            pref = ' '
 
        tmpl = (
 
        '%(pref)s<a class="%(cls)s" href="%(url)s">'
 
        '%(rev)s'
 
        '</a>'
 
        )
 
        return tmpl % {
 
         'pref': pref,
 
         'cls': 'revision-link',
 
         'url': url('changeset_home', repo_name=repository, revision=rev),
 
         'rev': rev,
 
        }
 

	
 
    newtext = URL_PAT.sub(url_func, text_)
 

	
 
    return newtext
 

	
 

	
 
def urlify_commit(text_, repository=None, link_=None):
 
    """
 
    Parses given text message and makes proper links.
 
    issues are linked to given issue-server, and rest is a changeset link
 
    if link_ is given, in other case it's a plain text
 

	
 
    :param text_:
 
    :param repository:
 
    :param link_: changeset link
 
    """
 
    import re
 
    import traceback
 

	
 
    # urlify changesets
 
    text_ = urlify_changesets(text_, repository)
 
    def escaper(string):
 
        return string.replace('<', '&lt;').replace('>', '&gt;')
 

	
 
    def linkify_others(t, l):
 
        urls = re.compile(r'(\<a.*?\<\/a\>)',)
 
        links = []
 
        for e in urls.split(t):
 
            if not urls.match(e):
 
                links.append('<a class="message-link" href="%s">%s</a>' % (l, e))
 
            else:
 
                links.append(e)
 

	
 
        return ''.join(links)
 

	
 

	
 
    # urlify changesets - extrac revisions and make link out of them
 
    text_ = urlify_changesets(escaper(text_), repository)
 

	
 
    try:
 
        conf = config['app_conf']
 

	
 
        URL_PAT = re.compile(r'%s' % conf.get('issue_pat'))
 

	
 
        if URL_PAT:
 
            ISSUE_SERVER_LNK = conf.get('issue_server_link')
 
            ISSUE_PREFIX = conf.get('issue_prefix')
 

	
 
            def url_func(match_obj):
 
                pref = ''
 
                if match_obj.group().startswith(' '):
 
                    pref = ' '
 

	
 
                issue_id = ''.join(match_obj.groups())
 
                tmpl = (
 
                '%(pref)s<a class="%(cls)s" href="%(url)s">'
 
                '%(issue-prefix)s%(id-repr)s'
 
                '</a>'
 
                )
 
                url = ISSUE_SERVER_LNK.replace('{id}', issue_id)
 
                if repository:
 
                    url = url.replace('{repo}', repository)
 

	
rhodecode/lib/hooks.py
Show inline comments
 
@@ -6,150 +6,169 @@
 
    Hooks runned by rhodecode
 

	
 
    :created_on: Aug 6, 2010
 
    :author: marcink
 
    :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>
 
    :license: GPLv3, see COPYING for more details.
 
"""
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
import os
 
import sys
 

	
 
from mercurial.scmutil import revrange
 
from mercurial.node import nullrev
 

	
 
from rhodecode import EXTENSIONS
 
from rhodecode.lib import helpers as h
 
from rhodecode.lib.utils import action_logger
 
from inspect import isfunction
 

	
 

	
 
def repo_size(ui, repo, hooktype=None, **kwargs):
 
    """
 
    Presents size of repository after push
 

	
 
    :param ui:
 
    :param repo:
 
    :param hooktype:
 
    """
 

	
 
    size_hg, size_root = 0, 0
 
    for path, dirs, files in os.walk(repo.root):
 
        if path.find('.hg') != -1:
 
            for f in files:
 
                try:
 
                    size_hg += os.path.getsize(os.path.join(path, f))
 
                except OSError:
 
                    pass
 
        else:
 
            for f in files:
 
                try:
 
                    size_root += os.path.getsize(os.path.join(path, f))
 
                except OSError:
 
                    pass
 

	
 
    size_hg_f = h.format_byte_size(size_hg)
 
    size_root_f = h.format_byte_size(size_root)
 
    size_total_f = h.format_byte_size(size_root + size_hg)
 

	
 
    last_cs = repo[len(repo) - 1]
 

	
 
    msg = ('Repository size .hg:%s repo:%s total:%s\n'
 
           'Last revision is now r%s:%s\n') % (
 
        size_hg_f, size_root_f, size_total_f, last_cs.rev(), last_cs.hex()[:12]
 
    )
 

	
 
    sys.stdout.write(msg)
 

	
 

	
 
def log_pull_action(ui, repo, **kwargs):
 
    """
 
    Logs user last pull action
 

	
 
    :param ui:
 
    :param repo:
 
    """
 

	
 
    extra_params = dict(repo.ui.configitems('rhodecode_extras'))
 
    username = extra_params['username']
 
    repository = extra_params['repository']
 
    extras = dict(repo.ui.configitems('rhodecode_extras'))
 
    username = extras['username']
 
    repository = extras['repository']
 
    action = 'pull'
 

	
 
    action_logger(username, action, repository, extra_params['ip'],
 
                  commit=True)
 
    action_logger(username, action, repository, extras['ip'], commit=True)
 
    # extension hook call
 
    callback = getattr(EXTENSIONS, 'PULL_HOOK', None)
 

	
 
    if isfunction(callback):
 
        kw = {}
 
        kw.update(extras)
 
        callback(**kw)
 
    return 0
 

	
 

	
 
def log_push_action(ui, repo, **kwargs):
 
    """
 
    Maps user last push action to new changeset id, from mercurial
 

	
 
    :param ui:
 
    :param repo:
 
    """
 

	
 
    extra_params = dict(repo.ui.configitems('rhodecode_extras'))
 
    username = extra_params['username']
 
    repository = extra_params['repository']
 
    action = extra_params['action'] + ':%s'
 
    extras = dict(repo.ui.configitems('rhodecode_extras'))
 
    username = extras['username']
 
    repository = extras['repository']
 
    action = extras['action'] + ':%s'
 
    node = kwargs['node']
 

	
 
    def get_revs(repo, rev_opt):
 
        if rev_opt:
 
            revs = revrange(repo, rev_opt)
 

	
 
            if len(revs) == 0:
 
                return (nullrev, nullrev)
 
            return (max(revs), min(revs))
 
        else:
 
            return (len(repo) - 1, 0)
 

	
 
    stop, start = get_revs(repo, [node + ':'])
 

	
 
    revs = (str(repo[r]) for r in xrange(start, stop + 1))
 

	
 
    action = action % ','.join(revs)
 

	
 
    action_logger(username, action, repository, extra_params['ip'],
 
                  commit=True)
 
    action_logger(username, action, repository, extras['ip'], commit=True)
 

	
 
    # extension hook call
 
    callback = getattr(EXTENSIONS, 'PUSH_HOOK', None)
 
    if isfunction(callback):
 
        kw = {'pushed_revs': revs}
 
        kw.update(extras)
 
        callback(**kw)
 
    return 0
 

	
 

	
 
def log_create_repository(repository_dict, created_by, **kwargs):
 
    """
 
    Post create repository Hook. This is a dummy function for admins to re-use
 
    if needed
 
    if needed. It's taken from rhodecode-extensions module and executed
 
    if present
 

	
 
    :param repository: dict dump of repository object
 
    :param created_by: username who created repository
 
    :param created_date: date of creation
 

	
 
    available keys of repository_dict:
 

	
 
     'repo_type',
 
     'description',
 
     'private',
 
     'created_on',
 
     'enable_downloads',
 
     'repo_id',
 
     'user_id',
 
     'enable_statistics',
 
     'clone_uri',
 
     'fork_id',
 
     'group_id',
 
     'repo_name'
 

	
 
    """
 

	
 
    callback = getattr(EXTENSIONS, 'CREATE_REPO_HOOK', None)
 
    if isfunction(callback):
 
        kw = {}
 
        kw.update(repository_dict)
 
        kw.update({'created_by': created_by})
 
        kw.update(kwargs)
 
        return callback(**kw)
 

	
 
    return 0
rhodecode/lib/indexers/__init__.py
Show inline comments
 
@@ -4,129 +4,127 @@
 
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
    Whoosh indexing module for RhodeCode
 

	
 
    :created_on: Aug 17, 2010
 
    :author: marcink
 
    :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>
 
    :license: GPLv3, see COPYING for more details.
 
"""
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
import os
 
import sys
 
import traceback
 
import logging
 
from os.path import dirname as dn, join as jn
 

	
 
#to get the rhodecode import
 
sys.path.append(dn(dn(dn(os.path.realpath(__file__)))))
 

	
 
from string import strip
 
from shutil import rmtree
 

	
 
from whoosh.analysis import RegexTokenizer, LowercaseFilter, StopFilter
 
from whoosh.fields import TEXT, ID, STORED, Schema, FieldType
 
from whoosh.index import create_in, open_dir
 
from whoosh.formats import Characters
 
from whoosh.highlight import highlight, HtmlFormatter, ContextFragmenter
 

	
 
from webhelpers.html.builder import escape
 
from sqlalchemy import engine_from_config
 

	
 
from rhodecode.model import init_model
 
from rhodecode.model.scm import ScmModel
 
from rhodecode.model.repo import RepoModel
 
from rhodecode.config.environment import load_environment
 
from rhodecode.lib import LANGUAGES_EXTENSIONS_MAP, LazyProperty
 
from rhodecode.lib.utils import BasePasterCommand, Command, add_cache
 

	
 
# EXTENSIONS WE WANT TO INDEX CONTENT OFF
 
INDEX_EXTENSIONS = LANGUAGES_EXTENSIONS_MAP.keys()
 
from rhodecode.lib.utils2 import LazyProperty
 
from rhodecode.lib.utils import BasePasterCommand, Command, add_cache,\
 
    load_rcextensions
 

	
 
# CUSTOM ANALYZER wordsplit + lowercase filter
 
ANALYZER = RegexTokenizer(expression=r"\w+") | LowercaseFilter()
 

	
 

	
 
#INDEX SCHEMA DEFINITION
 
SCHEMA = Schema(
 
    owner=TEXT(),
 
    repository=TEXT(stored=True),
 
    path=TEXT(stored=True),
 
    content=FieldType(format=Characters(), analyzer=ANALYZER,
 
                      scorable=True, stored=True),
 
    modtime=STORED(),
 
    extension=TEXT(stored=True)
 
)
 

	
 
IDX_NAME = 'HG_INDEX'
 
FORMATTER = HtmlFormatter('span', between='\n<span class="break">...</span>\n')
 
FRAGMENTER = ContextFragmenter(200)
 

	
 

	
 
class MakeIndex(BasePasterCommand):
 

	
 
    max_args = 1
 
    min_args = 1
 

	
 
    usage = "CONFIG_FILE"
 
    summary = "Creates index for full text search given configuration file"
 
    group_name = "RhodeCode"
 
    takes_config_file = -1
 
    parser = Command.standard_parser(verbose=True)
 

	
 
    def command(self):
 

	
 
        logging.config.fileConfig(self.path_to_ini_file)
 
        from pylons import config
 
        add_cache(config)
 
        engine = engine_from_config(config, 'sqlalchemy.db1.')
 
        init_model(engine)
 

	
 
        index_location = config['index_dir']
 
        repo_location = self.options.repo_location \
 
            if self.options.repo_location else RepoModel().repos_path
 
        repo_list = map(strip, self.options.repo_list.split(',')) \
 
            if self.options.repo_list else None
 

	
 
        load_rcextensions(config['here'])
 
        #======================================================================
 
        # WHOOSH DAEMON
 
        #======================================================================
 
        from rhodecode.lib.pidlock import LockHeld, DaemonLock
 
        from rhodecode.lib.indexers.daemon import WhooshIndexingDaemon
 
        try:
 
            l = DaemonLock(file_=jn(dn(dn(index_location)), 'make_index.lock'))
 
            WhooshIndexingDaemon(index_location=index_location,
 
                                 repo_location=repo_location,
 
                                 repo_list=repo_list)\
 
                                 repo_list=repo_list,)\
 
                .run(full_index=self.options.full_index)
 
            l.release()
 
        except LockHeld:
 
            sys.exit(1)
 

	
 
    def update_parser(self):
 
        self.parser.add_option('--repo-location',
 
                          action='store',
 
                          dest='repo_location',
 
                          help="Specifies repositories location to index OPTIONAL",
 
                          )
 
        self.parser.add_option('--index-only',
 
                          action='store',
 
                          dest='repo_list',
 
                          help="Specifies a comma separated list of repositores "
 
                                "to build index on OPTIONAL",
 
                          )
 
        self.parser.add_option('-f',
 
                          action='store_true',
 
                          dest='full_index',
 
                          help="Specifies that index should be made full i.e"
 
                                " destroy old and build from scratch",
 
                          default=False)
 

	
rhodecode/lib/indexers/daemon.py
Show inline comments
 
@@ -17,219 +17,221 @@
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

	
 
import os
 
import sys
 
import logging
 
import traceback
 

	
 
from shutil import rmtree
 
from time import mktime
 

	
 
from os.path import dirname as dn
 
from os.path import join as jn
 

	
 
#to get the rhodecode import
 
project_path = dn(dn(dn(dn(os.path.realpath(__file__)))))
 
sys.path.append(project_path)
 

	
 

	
 
from rhodecode.config.conf import INDEX_EXTENSIONS
 
from rhodecode.model.scm import ScmModel
 
from rhodecode.lib import safe_unicode
 
from rhodecode.lib.indexers import INDEX_EXTENSIONS, SCHEMA, IDX_NAME
 
from rhodecode.lib.utils2 import safe_unicode
 
from rhodecode.lib.indexers import SCHEMA, IDX_NAME
 

	
 
from rhodecode.lib.vcs.exceptions import ChangesetError, RepositoryError, \
 
    NodeDoesNotExistError
 

	
 
from whoosh.index import create_in, open_dir
 

	
 

	
 
log = logging.getLogger('whooshIndexer')
 
# create logger
 
log.setLevel(logging.DEBUG)
 
log.propagate = False
 
# create console handler and set level to debug
 
ch = logging.StreamHandler()
 
ch.setLevel(logging.DEBUG)
 

	
 
# create formatter
 
formatter = logging.Formatter("%(asctime)s - %(name)s -"
 
                              " %(levelname)s - %(message)s")
 

	
 
# add formatter to ch
 
ch.setFormatter(formatter)
 

	
 
# add ch to logger
 
log.addHandler(ch)
 
log = logging.getLogger('whoosh_indexer')
 

	
 

	
 
class WhooshIndexingDaemon(object):
 
    """
 
    Daemon for atomic jobs
 
    """
 

	
 
    def __init__(self, indexname=IDX_NAME, index_location=None,
 
                 repo_location=None, sa=None, repo_list=None):
 
        self.indexname = indexname
 

	
 
        self.index_location = index_location
 
        if not index_location:
 
            raise Exception('You have to provide index location')
 

	
 
        self.repo_location = repo_location
 
        if not repo_location:
 
            raise Exception('You have to provide repositories location')
 

	
 
        self.repo_paths = ScmModel(sa).repo_scan(self.repo_location)
 

	
 
        if repo_list:
 
            filtered_repo_paths = {}
 
            for repo_name, repo in self.repo_paths.items():
 
                if repo_name in repo_list:
 
                    filtered_repo_paths[repo_name] = repo
 

	
 
            self.repo_paths = filtered_repo_paths
 

	
 
        self.initial = False
 
        if not os.path.isdir(self.index_location):
 
            os.makedirs(self.index_location)
 
            log.info('Cannot run incremental index since it does not'
 
                     ' yet exist running full build')
 
            self.initial = True
 

	
 
    def get_paths(self, repo):
 
        """recursive walk in root dir and return a set of all path in that dir
 
        """
 
        recursive walk in root dir and return a set of all path in that dir
 
        based on repository walk function
 
        """
 
        index_paths_ = set()
 
        try:
 
            tip = repo.get_changeset('tip')
 
            for topnode, dirs, files in tip.walk('/'):
 
                for f in files:
 
                    index_paths_.add(jn(repo.path, f.path))
 

	
 
        except RepositoryError, e:
 
            log.debug(traceback.format_exc())
 
            pass
 
        return index_paths_
 

	
 
    def get_node(self, repo, path):
 
        n_path = path[len(repo.path) + 1:]
 
        node = repo.get_changeset().get_node(n_path)
 
        return node
 

	
 
    def get_node_mtime(self, node):
 
        return mktime(node.last_changeset.date.timetuple())
 

	
 
    def add_doc(self, writer, path, repo, repo_name):
 
        """Adding doc to writer this function itself fetches data from
 
        the instance of vcs backend"""
 
        node = self.get_node(repo, path)
 
        """
 
        Adding doc to writer this function itself fetches data from
 
        the instance of vcs backend
 
        """
 

	
 
        #we just index the content of chosen files, and skip binary files
 
        node = self.get_node(repo, path)
 
        indexed = indexed_w_content = 0
 
        # we just index the content of chosen files, and skip binary files
 
        if node.extension in INDEX_EXTENSIONS and not node.is_binary:
 

	
 
            u_content = node.content
 
            if not isinstance(u_content, unicode):
 
                log.warning('  >> %s Could not get this content as unicode '
 
                          'replacing with empty content', path)
 
                            'replacing with empty content' % path)
 
                u_content = u''
 
            else:
 
                log.debug('    >> %s [WITH CONTENT]' % path)
 
                indexed_w_content += 1
 

	
 
        else:
 
            log.debug('    >> %s' % path)
 
            #just index file name without it's content
 
            # just index file name without it's content
 
            u_content = u''
 
            indexed += 1
 

	
 
        writer.add_document(owner=unicode(repo.contact),
 
                        repository=safe_unicode(repo_name),
 
                        path=safe_unicode(path),
 
                        content=u_content,
 
                        modtime=self.get_node_mtime(node),
 
                        extension=node.extension)
 
        writer.add_document(
 
            owner=unicode(repo.contact),
 
            repository=safe_unicode(repo_name),
 
            path=safe_unicode(path),
 
            content=u_content,
 
            modtime=self.get_node_mtime(node),
 
            extension=node.extension
 
        )
 
        return indexed, indexed_w_content
 

	
 
    def build_index(self):
 
        if os.path.exists(self.index_location):
 
            log.debug('removing previous index')
 
            rmtree(self.index_location)
 

	
 
        if not os.path.exists(self.index_location):
 
            os.mkdir(self.index_location)
 

	
 
        idx = create_in(self.index_location, SCHEMA, indexname=IDX_NAME)
 
        writer = idx.writer()
 

	
 
        log.debug('BUILDIN INDEX FOR EXTENSIONS %s' % INDEX_EXTENSIONS)
 
        for repo_name, repo in self.repo_paths.items():
 
            log.debug('building index @ %s' % repo.path)
 

	
 
            i_cnt = iwc_cnt = 0
 
            for idx_path in self.get_paths(repo):
 
                self.add_doc(writer, idx_path, repo, repo_name)
 
                i, iwc = self.add_doc(writer, idx_path, repo, repo_name)
 
                i_cnt += i
 
                iwc_cnt += iwc
 
            log.debug('added %s files %s with content for repo %s' % (
 
                         i_cnt + iwc_cnt, iwc_cnt, repo.path)
 
            )
 

	
 
        log.debug('>> COMMITING CHANGES <<')
 
        writer.commit(merge=True)
 
        log.debug('>>> FINISHED BUILDING INDEX <<<')
 

	
 
    def update_index(self):
 
        log.debug('STARTING INCREMENTAL INDEXING UPDATE')
 
        log.debug('STARTING INCREMENTAL INDEXING UPDATE FOR EXTENSIONS %s' %
 
                  INDEX_EXTENSIONS)
 

	
 
        idx = open_dir(self.index_location, indexname=self.indexname)
 
        # The set of all paths in the index
 
        indexed_paths = set()
 
        # The set of all paths we need to re-index
 
        to_index = set()
 

	
 
        reader = idx.reader()
 
        writer = idx.writer()
 

	
 
        # Loop over the stored fields in the index
 
        for fields in reader.all_stored_fields():
 
            indexed_path = fields['path']
 
            indexed_paths.add(indexed_path)
 

	
 
            repo = self.repo_paths[fields['repository']]
 

	
 
            try:
 
                node = self.get_node(repo, indexed_path)
 
            except (ChangesetError, NodeDoesNotExistError):
 
                # This file was deleted since it was indexed
 
                log.debug('removing from index %s' % indexed_path)
 
                writer.delete_by_term('path', indexed_path)
 

	
 
            else:
 
                # Check if this file was changed since it was indexed
 
                indexed_time = fields['modtime']
 
                mtime = self.get_node_mtime(node)
 
                if mtime > indexed_time:
 
                    # The file has changed, delete it and add it to the list of
 
                    # files to reindex
 
                    log.debug('adding to reindex list %s' % indexed_path)
 
                    writer.delete_by_term('path', indexed_path)
 
                    to_index.add(indexed_path)
 

	
 
        # Loop over the files in the filesystem
 
        # Assume we have a function that gathers the filenames of the
 
        # documents to be indexed
 
        ri_cnt = riwc_cnt = 0
 
        for repo_name, repo in self.repo_paths.items():
 
            for path in self.get_paths(repo):
 
                if path in to_index or path not in indexed_paths:
 
                    # This is either a file that's changed, or a new file
 
                    # that wasn't indexed before. So index it!
 
                    self.add_doc(writer, path, repo, repo_name)
 
                    i, iwc = self.add_doc(writer, path, repo, repo_name)
 
                    log.debug('re indexing %s' % path)
 

	
 
                    ri_cnt += i
 
                    riwc_cnt += iwc
 
        log.debug('added %s files %s with content for repo %s' % (
 
                     ri_cnt + riwc_cnt, riwc_cnt, repo.path)
 
        )
 
        log.debug('>> COMMITING CHANGES <<')
 
        writer.commit(merge=True)
 
        log.debug('>>> FINISHED REBUILDING INDEX <<<')
 

	
 
    def run(self, full_index=False):
 
        """Run daemon"""
 
        if full_index or self.initial:
 
            self.build_index()
 
        else:
 
            self.update_index()
rhodecode/lib/markup_renderer.py
Show inline comments
 
@@ -6,49 +6,49 @@
 

	
 
    Renderer for markup languages with ability to parse using rst or markdown
 

	
 
    :created_on: Oct 27, 2011
 
    :author: marcink
 
    :copyright: (C) 2011-2012 Marcin Kuzminski <marcin@python-works.com>
 
    :license: GPLv3, see COPYING for more details.
 
"""
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

	
 
import re
 
import logging
 

	
 
from rhodecode.lib import safe_unicode
 
from rhodecode.lib.utils2 import safe_unicode
 

	
 
log = logging.getLogger(__name__)
 

	
 

	
 
class MarkupRenderer(object):
 
    RESTRUCTUREDTEXT_DISALLOWED_DIRECTIVES = ['include', 'meta', 'raw']
 

	
 
    MARKDOWN_PAT = re.compile(r'md|mkdn?|mdown|markdown', re.IGNORECASE)
 
    RST_PAT = re.compile(r're?st', re.IGNORECASE)
 
    PLAIN_PAT = re.compile(r'readme', re.IGNORECASE)
 

	
 
    def __detect_renderer(self, source, filename=None):
 
        """
 
        runs detection of what renderer should be used for generating html
 
        from a markup language
 

	
 
        filename can be also explicitly a renderer name
 

	
 
        :param source:
 
        :param filename:
 
        """
 

	
 
        if MarkupRenderer.MARKDOWN_PAT.findall(filename):
 
            detected_renderer = 'markdown'
rhodecode/lib/middleware/https_fixup.py
Show inline comments
 
@@ -2,49 +2,49 @@
 
"""
 
    rhodecode.lib.middleware.https_fixup
 
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
    middleware to handle https correctly
 

	
 
    :created_on: May 23, 2010
 
    :author: marcink
 
    :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>
 
    :license: GPLv3, see COPYING for more details.
 
"""
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

	
 
from rhodecode.lib import str2bool
 
from rhodecode.lib.utils2 import str2bool
 

	
 

	
 
class HttpsFixup(object):
 

	
 
    def __init__(self, app, config):
 
        self.application = app
 
        self.config = config
 

	
 
    def __call__(self, environ, start_response):
 
        self.__fixup(environ)
 
        return self.application(environ, start_response)
 

	
 
    def __fixup(self, environ):
 
        """
 
        Function to fixup the environ as needed. In order to use this
 
        middleware you should set this header inside your
 
        proxy ie. nginx, apache etc.
 
        """
 

	
 
        if str2bool(self.config.get('force_https')):
 
            proto = 'https'
 
        else:
 
            if 'HTTP_X_URL_SCHEME' in environ:
 
                proto = environ.get('HTTP_X_URL_SCHEME')
rhodecode/lib/middleware/simplegit.py
Show inline comments
 
@@ -44,105 +44,110 @@ class SimpleGitUploadPackHandler(dulserv
 
          graph_walker.determine_wants, graph_walker, self.progress,
 
          get_tagged=self.get_tagged)
 

	
 
        # Do they want any objects?
 
        if objects_iter is None or len(objects_iter) == 0:
 
            return
 

	
 
        self.progress("counting objects: %d, done.\n" % len(objects_iter))
 
        dulserver.write_pack_objects(dulserver.ProtocolFile(None, write),
 
                                  objects_iter, len(objects_iter))
 
        messages = []
 
        messages.append('thank you for using rhodecode')
 

	
 
        for msg in messages:
 
            self.progress(msg + "\n")
 
        # we are done
 
        self.proto.write("0000")
 

	
 
dulserver.DEFAULT_HANDLERS = {
 
  'git-upload-pack': SimpleGitUploadPackHandler,
 
  'git-receive-pack': dulserver.ReceivePackHandler,
 
}
 

	
 
from dulwich.repo import Repo
 
from dulwich.web import HTTPGitApplication
 
from dulwich.web import make_wsgi_chain
 

	
 
from paste.httpheaders import REMOTE_USER, AUTH_TYPE
 

	
 
from rhodecode.lib import safe_str
 
from rhodecode.lib.utils2 import safe_str
 
from rhodecode.lib.base import BaseVCSController
 
from rhodecode.lib.auth import get_container_username
 
from rhodecode.lib.utils import is_valid_repo
 
from rhodecode.model.db import User
 

	
 
from webob.exc import HTTPNotFound, HTTPForbidden, HTTPInternalServerError
 

	
 
log = logging.getLogger(__name__)
 

	
 

	
 
GIT_PROTO_PAT = re.compile(r'^/(.+)/(info/refs|git-upload-pack|git-receive-pack)')
 

	
 

	
 
def is_git(environ):
 
    path_info = environ['PATH_INFO']
 
    isgit_path = GIT_PROTO_PAT.match(path_info)
 
    log.debug('is a git path %s pathinfo : %s' % (isgit_path, path_info))
 
    log.debug('pathinfo: %s detected as GIT %s' % (
 
        path_info, isgit_path != None)
 
    )
 
    return isgit_path
 

	
 

	
 
class SimpleGit(BaseVCSController):
 

	
 
    def _handle_request(self, environ, start_response):
 

	
 
        if not is_git(environ):
 
            return self.application(environ, start_response)
 

	
 
        proxy_key = 'HTTP_X_REAL_IP'
 
        def_key = 'REMOTE_ADDR'
 
        ipaddr = environ.get(proxy_key, environ.get(def_key, '0.0.0.0'))
 
        username = None
 
        # skip passing error to error controller
 
        environ['pylons.status_code_redirect'] = True
 

	
 
        #======================================================================
 
        # EXTRACT REPOSITORY NAME FROM ENV
 
        #======================================================================
 
        try:
 
            repo_name = self.__get_repository(environ)
 
            log.debug('Extracted repo name is %s' % repo_name)
 
        except:
 
            return HTTPInternalServerError()(environ, start_response)
 

	
 
        # quick check if that dir exists...
 
        if is_valid_repo(repo_name, self.basepath) is False:
 
            return HTTPNotFound()(environ, start_response)
 

	
 
        #======================================================================
 
        # GET ACTION PULL or PUSH
 
        #======================================================================
 
        action = self.__get_action(environ)
 

	
 
        #======================================================================
 
        # CHECK ANONYMOUS PERMISSION
 
        #======================================================================
 

	
 
        if action in ['pull', 'push']:
 
            anonymous_user = self.__get_user('default')
 
            username = anonymous_user.username
 
            anonymous_perm = self._check_permission(action, anonymous_user,
 
                                                    repo_name)
 

	
 
            if anonymous_perm is not True or anonymous_user.active is False:
 
                if anonymous_perm is not True:
 
                    log.debug('Not enough credentials to access this '
 
                              'repository as anonymous user')
 
                if anonymous_user.active is False:
 
                    log.debug('Anonymous access is disabled, running '
 
                              'authentication')
 
                #==============================================================
 
                # DEFAULT PERM FAILED OR ANONYMOUS ACCESS IS DISABLED SO WE
 
                # NEED TO AUTHENTICATE AND ASK FOR AUTH USER PERMISSIONS
 
                #==============================================================
 

	
 
                # Attempting to retrieve username from the container
 
                username = get_container_username(environ, self.config)
 

	
 
                # If not authenticated by the container, running basic auth
 
                if not username:
 
                    self.authenticate.realm = \
 
@@ -156,76 +161,72 @@ class SimpleGit(BaseVCSController):
 
                        return result.wsgi_application(environ, start_response)
 

	
 
                #==============================================================
 
                # CHECK PERMISSIONS FOR THIS REQUEST USING GIVEN USERNAME
 
                #==============================================================
 
                if action in ['pull', 'push']:
 
                    try:
 
                        user = self.__get_user(username)
 
                        if user is None or not user.active:
 
                            return HTTPForbidden()(environ, start_response)
 
                        username = user.username
 
                    except:
 
                        log.error(traceback.format_exc())
 
                        return HTTPInternalServerError()(environ,
 
                                                         start_response)
 

	
 
                    #check permissions for this repository
 
                    perm = self._check_permission(action, user, repo_name)
 
                    if perm is not True:
 
                        return HTTPForbidden()(environ, start_response)
 

	
 
        #===================================================================
 
        # GIT REQUEST HANDLING
 
        #===================================================================
 
        repo_path = safe_str(os.path.join(self.basepath, repo_name))
 
        repo_path = os.path.join(safe_str(self.basepath), safe_str(repo_name))
 
        log.debug('Repository path is %s' % repo_path)
 

	
 
        # quick check if that dir exists...
 
        if is_valid_repo(repo_name, self.basepath) is False:
 
            return HTTPNotFound()(environ, start_response)
 

	
 
        try:
 
            #invalidate cache on push
 
            if action == 'push':
 
                self._invalidate_cache(repo_name)
 
            log.info('%s action on GIT repo "%s"' % (action, repo_name))
 
            app = self.__make_app(repo_name, repo_path)
 
            return app(environ, start_response)
 
        except Exception:
 
            log.error(traceback.format_exc())
 
            return HTTPInternalServerError()(environ, start_response)
 

	
 
    def __make_app(self, repo_name, repo_path):
 
        """
 
        Make an wsgi application using dulserver
 

	
 
        :param repo_name: name of the repository
 
        :param repo_path: full path to the repository
 
        """
 
        _d = {'/' + repo_name: Repo(repo_path)}
 
        backend = dulserver.DictBackend(_d)
 
        gitserve = HTTPGitApplication(backend)
 
        gitserve = make_wsgi_chain(backend)
 

	
 
        return gitserve
 

	
 
    def __get_repository(self, environ):
 
        """
 
        Get's repository name out of PATH_INFO header
 

	
 
        :param environ: environ where PATH_INFO is stored
 
        """
 
        try:
 
            environ['PATH_INFO'] = self._get_by_id(environ['PATH_INFO'])
 
            repo_name = GIT_PROTO_PAT.match(environ['PATH_INFO']).group(1)
 
        except:
 
            log.error(traceback.format_exc())
 
            raise
 

	
 
        return repo_name
 

	
 
    def __get_user(self, username):
 
        return User.get_by_username(username)
 

	
 
    def __get_action(self, environ):
 
        """
 
        Maps git request commands into a pull or push command.
rhodecode/lib/middleware/simplehg.py
Show inline comments
 
@@ -6,177 +6,188 @@
 
    SimpleHG middleware for handling mercurial protocol request
 
    (push/clone etc.). It's implemented with basic auth function
 

	
 
    :created_on: Apr 28, 2010
 
    :author: marcink
 
    :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>
 
    :license: GPLv3, see COPYING for more details.
 
"""
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

	
 
import os
 
import logging
 
import traceback
 
import urllib
 

	
 
from mercurial.error import RepoError
 
from mercurial.hgweb import hgweb_mod
 

	
 
from paste.httpheaders import REMOTE_USER, AUTH_TYPE
 

	
 
from rhodecode.lib import safe_str
 
from rhodecode.lib.utils2 import safe_str
 
from rhodecode.lib.base import BaseVCSController
 
from rhodecode.lib.auth import get_container_username
 
from rhodecode.lib.utils import make_ui, is_valid_repo, ui_sections
 
from rhodecode.model.db import User
 

	
 
from webob.exc import HTTPNotFound, HTTPForbidden, HTTPInternalServerError
 

	
 
log = logging.getLogger(__name__)
 

	
 

	
 
def is_mercurial(environ):
 
    """Returns True if request's target is mercurial server - header
 
    """
 
    Returns True if request's target is mercurial server - header
 
    ``HTTP_ACCEPT`` of such request would start with ``application/mercurial``.
 
    """
 
    http_accept = environ.get('HTTP_ACCEPT')
 
    path_info = environ['PATH_INFO']
 
    if http_accept and http_accept.startswith('application/mercurial'):
 
        return True
 
    return False
 
        ishg_path = True
 
    else:
 
        ishg_path = False
 

	
 
    log.debug('pathinfo: %s detected as HG %s' % (
 
        path_info, ishg_path)
 
    )
 
    return ishg_path
 

	
 

	
 
class SimpleHg(BaseVCSController):
 

	
 
    def _handle_request(self, environ, start_response):
 
        if not is_mercurial(environ):
 
            return self.application(environ, start_response)
 

	
 
        proxy_key = 'HTTP_X_REAL_IP'
 
        def_key = 'REMOTE_ADDR'
 
        ipaddr = environ.get(proxy_key, environ.get(def_key, '0.0.0.0'))
 

	
 
        # skip passing error to error controller
 
        environ['pylons.status_code_redirect'] = True
 

	
 
        #======================================================================
 
        # EXTRACT REPOSITORY NAME FROM ENV
 
        #======================================================================
 
        try:
 
            repo_name = environ['REPO_NAME'] = self.__get_repository(environ)
 
            log.debug('Extracted repo name is %s' % repo_name)
 
        except:
 
            return HTTPInternalServerError()(environ, start_response)
 

	
 
        # quick check if that dir exists...
 
        if is_valid_repo(repo_name, self.basepath) is False:
 
            return HTTPNotFound()(environ, start_response)
 

	
 
        #======================================================================
 
        # GET ACTION PULL or PUSH
 
        #======================================================================
 
        action = self.__get_action(environ)
 

	
 
        #======================================================================
 
        # CHECK ANONYMOUS PERMISSION
 
        #======================================================================
 
        if action in ['pull', 'push']:
 
            anonymous_user = self.__get_user('default')
 

	
 
            username = anonymous_user.username
 
            anonymous_perm = self._check_permission(action, anonymous_user,
 
                                                    repo_name)
 

	
 
            if anonymous_perm is not True or anonymous_user.active is False:
 
                if anonymous_perm is not True:
 
                    log.debug('Not enough credentials to access this '
 
                              'repository as anonymous user')
 
                if anonymous_user.active is False:
 
                    log.debug('Anonymous access is disabled, running '
 
                              'authentication')
 
                #==============================================================
 
                # DEFAULT PERM FAILED OR ANONYMOUS ACCESS IS DISABLED SO WE
 
                # NEED TO AUTHENTICATE AND ASK FOR AUTH USER PERMISSIONS
 
                #==============================================================
 

	
 
                # Attempting to retrieve username from the container
 
                username = get_container_username(environ, self.config)
 

	
 
                # If not authenticated by the container, running basic auth
 
                if not username:
 
                    self.authenticate.realm = \
 
                        safe_str(self.config['rhodecode_realm'])
 
                    result = self.authenticate(environ)
 
                    if isinstance(result, str):
 
                        AUTH_TYPE.update(environ, 'basic')
 
                        REMOTE_USER.update(environ, result)
 
                        username = result
 
                    else:
 
                        return result.wsgi_application(environ, start_response)
 

	
 
                #==============================================================
 
                # CHECK PERMISSIONS FOR THIS REQUEST USING GIVEN USERNAME
 
                #==============================================================
 
                if action in ['pull', 'push']:
 
                    try:
 
                        user = self.__get_user(username)
 
                        if user is None or not user.active:
 
                            return HTTPForbidden()(environ, start_response)
 
                        username = user.username
 
                    except:
 
                        log.error(traceback.format_exc())
 
                        return HTTPInternalServerError()(environ,
 
                                                         start_response)
 

	
 
                    #check permissions for this repository
 
                    perm = self._check_permission(action, user,
 
                                                   repo_name)
 
                    perm = self._check_permission(action, user, repo_name)
 
                    if perm is not True:
 
                        return HTTPForbidden()(environ, start_response)
 

	
 
        extras = {'ip': ipaddr,
 
                  'username': username,
 
                  'action': action,
 
                  'repository': repo_name}
 
        # extras are injected into mercurial UI object and later available
 
        # in hg hooks executed by rhodecode
 
        extras = {
 
            'ip': ipaddr,
 
            'username': username,
 
            'action': action,
 
            'repository': repo_name
 
        }
 

	
 
        #======================================================================
 
        # MERCURIAL REQUEST HANDLING
 
        #======================================================================
 

	
 
        repo_path = safe_str(os.path.join(self.basepath, repo_name))
 
        repo_path = os.path.join(safe_str(self.basepath), safe_str(repo_name))
 
        log.debug('Repository path is %s' % repo_path)
 

	
 
        baseui = make_ui('db')
 
        self.__inject_extras(repo_path, baseui, extras)
 

	
 
        # quick check if that dir exists...
 
        if is_valid_repo(repo_name, self.basepath) is False:
 
            return HTTPNotFound()(environ, start_response)
 

	
 
        try:
 
            # invalidate cache on push
 
            if action == 'push':
 
                self._invalidate_cache(repo_name)
 
            log.info('%s action on HG repo "%s"' % (action, repo_name))
 
            app = self.__make_app(repo_path, baseui, extras)
 
            return app(environ, start_response)
 
        except RepoError, e:
 
            if str(e).find('not found') != -1:
 
                return HTTPNotFound()(environ, start_response)
 
        except Exception:
 
            log.error(traceback.format_exc())
 
            return HTTPInternalServerError()(environ, start_response)
 

	
 
    def __make_app(self, repo_name, baseui, extras):
 
        """
 
        Make an wsgi application using hgweb, and inject generated baseui
 
        instance, additionally inject some extras into ui object
 
        """
 
        return hgweb_mod.hgweb(repo_name, name=repo_name, baseui=baseui)
 

	
 
    def __get_repository(self, environ):
 
        """
 
        Get's repository name out of PATH_INFO header
rhodecode/lib/utils.py
Show inline comments
 
@@ -30,77 +30,82 @@ import datetime
 
import traceback
 
import paste
 
import beaker
 
import tarfile
 
import shutil
 
from os.path import abspath
 
from os.path import dirname as dn, join as jn
 

	
 
from paste.script.command import Command, BadCommand
 

	
 
from mercurial import ui, config
 

	
 
from webhelpers.text import collapse, remove_formatting, strip_tags
 

	
 
from rhodecode.lib.vcs import get_backend
 
from rhodecode.lib.vcs.backends.base import BaseChangeset
 
from rhodecode.lib.vcs.utils.lazy import LazyProperty
 
from rhodecode.lib.vcs.utils.helpers import get_scm
 
from rhodecode.lib.vcs.exceptions import VCSError
 

	
 
from rhodecode.lib.caching_query import FromCache
 

	
 
from rhodecode.model import meta
 
from rhodecode.model.db import Repository, User, RhodeCodeUi, \
 
    UserLog, RepoGroup, RhodeCodeSetting, UserRepoGroupToPerm
 
    UserLog, RepoGroup, RhodeCodeSetting, UserRepoGroupToPerm,\
 
    CacheInvalidation
 
from rhodecode.model.meta import Session
 
from rhodecode.model.repos_group import ReposGroupModel
 
from rhodecode.lib.utils2 import safe_str, safe_unicode
 
from rhodecode.lib.vcs.utils.fakemod import create_module
 

	
 
log = logging.getLogger(__name__)
 

	
 
REMOVED_REPO_PAT = re.compile(r'rm__\d{8}_\d{6}_\d{6}__.*')
 

	
 

	
 
def recursive_replace(str_, replace=' '):
 
    """Recursive replace of given sign to just one instance
 
    """
 
    Recursive replace of given sign to just one instance
 

	
 
    :param str_: given string
 
    :param replace: char to find and replace multiple instances
 

	
 
    Examples::
 
    >>> recursive_replace("Mighty---Mighty-Bo--sstones",'-')
 
    'Mighty-Mighty-Bo-sstones'
 
    """
 

	
 
    if str_.find(replace * 2) == -1:
 
        return str_
 
    else:
 
        str_ = str_.replace(replace * 2, replace)
 
        return recursive_replace(str_, replace)
 

	
 

	
 
def repo_name_slug(value):
 
    """Return slug of name of repository
 
    """
 
    Return slug of name of repository
 
    This function is called on each creation/modification
 
    of repository to prevent bad names in repo
 
    """
 

	
 
    slug = remove_formatting(value)
 
    slug = strip_tags(slug)
 

	
 
    for c in """=[]\;'"<>,/~!@#$%^&*()+{}|: """:
 
        slug = slug.replace(c, '-')
 
    slug = recursive_replace(slug, '-')
 
    slug = collapse(slug, '-')
 
    return slug
 

	
 

	
 
def get_repo_slug(request):
 
    _repo = request.environ['pylons.routes_dict'].get('repo_name')
 
    if _repo:
 
        _repo = _repo.rstrip('/')
 
    return _repo
 

	
 

	
 
def get_repos_group_slug(request):
 
    _group = request.environ['pylons.routes_dict'].get('group_name')
 
    if _group:
 
@@ -133,153 +138,158 @@ def action_logger(user, action, repo, ip
 
            user_obj = User.get_by_username(user)
 
        else:
 
            raise Exception('You have to provide user object or username')
 

	
 
        if hasattr(repo, 'repo_id'):
 
            repo_obj = Repository.get(repo.repo_id)
 
            repo_name = repo_obj.repo_name
 
        elif  isinstance(repo, basestring):
 
            repo_name = repo.lstrip('/')
 
            repo_obj = Repository.get_by_repo_name(repo_name)
 
        else:
 
            raise Exception('You have to provide repository to action logger')
 

	
 
        user_log = UserLog()
 
        user_log.user_id = user_obj.user_id
 
        user_log.action = action
 

	
 
        user_log.repository_id = repo_obj.repo_id
 
        user_log.repository_name = repo_name
 

	
 
        user_log.action_date = datetime.datetime.now()
 
        user_log.user_ip = ipaddr
 
        sa.add(user_log)
 

	
 
        log.info('Adding user %s, action %s on %s' % (user_obj, action, repo))
 
        log.info(
 
            'Adding user %s, action %s on %s' % (user_obj, action,
 
                                                 safe_unicode(repo))
 
        )
 
        if commit:
 
            sa.commit()
 
    except:
 
        log.error(traceback.format_exc())
 
        raise
 

	
 

	
 
def get_repos(path, recursive=False):
 
    """
 
    Scans given path for repos and return (name,(type,path)) tuple
 

	
 
    :param path: path to scan for repositories
 
    :param recursive: recursive search and return names with subdirs in front
 
    """
 

	
 
    # remove ending slash for better results
 
    path = path.rstrip(os.sep)
 

	
 
    def _get_repos(p):
 
        if not os.access(p, os.W_OK):
 
            return
 
        for dirpath in os.listdir(p):
 
            if os.path.isfile(os.path.join(p, dirpath)):
 
                continue
 
            cur_path = os.path.join(p, dirpath)
 
            try:
 
                scm_info = get_scm(cur_path)
 
                yield scm_info[1].split(path, 1)[-1].lstrip(os.sep), scm_info
 
            except VCSError:
 
                if not recursive:
 
                    continue
 
                #check if this dir containts other repos for recursive scan
 
                rec_path = os.path.join(p, dirpath)
 
                if os.path.isdir(rec_path):
 
                    for inner_scm in _get_repos(rec_path):
 
                        yield inner_scm
 

	
 
    return _get_repos(path)
 

	
 

	
 
def is_valid_repo(repo_name, base_path):
 
    """
 
    Returns True if given path is a valid repository False otherwise
 

	
 
    :param repo_name:
 
    :param base_path:
 

	
 
    :return True: if given path is a valid repository
 
    """
 
    full_path = os.path.join(base_path, repo_name)
 
    full_path = os.path.join(safe_str(base_path), safe_str(repo_name))
 

	
 
    try:
 
        get_scm(full_path)
 
        return True
 
    except VCSError:
 
        return False
 

	
 

	
 
def is_valid_repos_group(repos_group_name, base_path):
 
    """
 
    Returns True if given path is a repos group False otherwise
 

	
 
    :param repo_name:
 
    :param base_path:
 
    """
 
    full_path = os.path.join(base_path, repos_group_name)
 
    full_path = os.path.join(safe_str(base_path), safe_str(repos_group_name))
 

	
 
    # check if it's not a repo
 
    if is_valid_repo(repos_group_name, base_path):
 
        return False
 

	
 
    # check if it's a valid path
 
    if os.path.isdir(full_path):
 
        return True
 

	
 
    return False
 

	
 

	
 
def ask_ok(prompt, retries=4, complaint='Yes or no, please!'):
 
    while True:
 
        ok = raw_input(prompt)
 
        if ok in ('y', 'ye', 'yes'):
 
            return True
 
        if ok in ('n', 'no', 'nop', 'nope'):
 
            return False
 
        retries = retries - 1
 
        if retries < 0:
 
            raise IOError
 
        print complaint
 

	
 
#propagated from mercurial documentation
 
ui_sections = ['alias', 'auth',
 
                'decode/encode', 'defaults',
 
                'diff', 'email',
 
                'extensions', 'format',
 
                'merge-patterns', 'merge-tools',
 
                'hooks', 'http_proxy',
 
                'smtp', 'patch',
 
                'paths', 'profiling',
 
                'server', 'trusted',
 
                'ui', 'web', ]
 

	
 

	
 
def make_ui(read_from='file', path=None, checkpaths=True):
 
    """A function that will read python rc files or database
 
    """
 
    A function that will read python rc files or database
 
    and make an mercurial ui object from read options
 

	
 
    :param path: path to mercurial config file
 
    :param checkpaths: check the path
 
    :param read_from: read from 'file' or 'db'
 
    """
 

	
 
    baseui = ui.ui()
 

	
 
    # clean the baseui object
 
    baseui._ocfg = config.config()
 
    baseui._ucfg = config.config()
 
    baseui._tcfg = config.config()
 

	
 
    if read_from == 'file':
 
        if not os.path.isfile(path):
 
            log.debug('hgrc file is not present at %s skipping...' % path)
 
            return False
 
        log.debug('reading hgrc from %s' % path)
 
        cfg = config.config()
 
        cfg.read(path)
 
        for section in ui_sections:
 
            for k, v in cfg.items(section):
 
                log.debug('settings ui from file[%s]%s:%s' % (section, k, v))
 
@@ -350,161 +360,188 @@ class EmptyChangeset(BaseChangeset):
 
        Returns raw string identifying this changeset, useful for web
 
        representation.
 
        """
 

	
 
        return self._empty_cs
 

	
 
    @LazyProperty
 
    def branch(self):
 
        return get_backend(self.alias).DEFAULT_BRANCH_NAME
 

	
 
    @LazyProperty
 
    def short_id(self):
 
        return self.raw_id[:12]
 

	
 
    def get_file_changeset(self, path):
 
        return self
 

	
 
    def get_file_content(self, path):
 
        return u''
 

	
 
    def get_file_size(self, path):
 
        return 0
 

	
 

	
 
def map_groups(groups):
 
def map_groups(path):
 
    """
 
    Checks for groups existence, and creates groups structures.
 
    It returns last group in structure
 
    Given a full path to a repository, create all nested groups that this
 
    repo is inside. This function creates parent-child relationships between
 
    groups and creates default perms for all new groups.
 

	
 
    :param groups: list of groups structure
 
    :param paths: full path to repository
 
    """
 
    sa = meta.Session
 

	
 
    groups = path.split(Repository.url_sep())
 
    parent = None
 
    group = None
 

	
 
    # last element is repo in nested groups structure
 
    groups = groups[:-1]
 
    rgm = ReposGroupModel(sa)
 
    for lvl, group_name in enumerate(groups):
 
        group_name = '/'.join(groups[:lvl] + [group_name])
 
        group = RepoGroup.get_by_group_name(group_name)
 
        desc = '%s group' % group_name
 

	
 
#        # WTF that doesn't work !?
 
#        if group is None:
 
#            group = rgm.create(group_name, desc, parent, just_db=True)
 
#            sa.commit()
 

	
 
        # skip folders that are now removed repos
 
        if REMOVED_REPO_PAT.match(group_name):
 
            break
 

	
 
        if group is None:
 
            log.debug('creating group level: %s group_name: %s' % (lvl, group_name))
 
            log.debug('creating group level: %s group_name: %s' % (lvl,
 
                                                                   group_name))
 
            group = RepoGroup(group_name, parent)
 
            group.group_description = desc
 
            sa.add(group)
 
            rgm._create_default_perms(group)
 
            sa.commit()
 
            sa.flush()
 
        parent = group
 
    return group
 

	
 

	
 
def repo2db_mapper(initial_repo_list, remove_obsolete=False):
 
    """
 
    maps all repos given in initial_repo_list, non existing repositories
 
    are created, if remove_obsolete is True it also check for db entries
 
    that are not in initial_repo_list and removes them.
 

	
 
    :param initial_repo_list: list of repositories found by scanning methods
 
    :param remove_obsolete: check for obsolete entries in database
 
    """
 
    from rhodecode.model.repo import RepoModel
 
    sa = meta.Session
 
    rm = RepoModel()
 
    user = sa.query(User).filter(User.admin == True).first()
 
    if user is None:
 
        raise Exception('Missing administrative account !')
 
    added = []
 

	
 
    for name, repo in initial_repo_list.items():
 
        group = map_groups(name.split(Repository.url_sep()))
 
        group = map_groups(name)
 
        if not rm.get_by_repo_name(name, cache=False):
 
            log.info('repository %s not found creating default' % name)
 
            added.append(name)
 
            form_data = {
 
             'repo_name': name,
 
             'repo_name_full': name,
 
             'repo_type': repo.alias,
 
             'description': repo.description \
 
                if repo.description != 'unknown' else '%s repository' % name,
 
             'private': False,
 
             'group_id': getattr(group, 'group_id', None)
 
            }
 
            rm.create(form_data, user, just_db=True)
 
    sa.commit()
 
    removed = []
 
    if remove_obsolete:
 
        #remove from database those repositories that are not in the filesystem
 
        # remove from database those repositories that are not in the filesystem
 
        for repo in sa.query(Repository).all():
 
            if repo.repo_name not in initial_repo_list.keys():
 
                log.debug("Removing non existing repository found in db %s" %
 
                          repo.repo_name)
 
                removed.append(repo.repo_name)
 
                sa.delete(repo)
 
                sa.commit()
 

	
 
    # clear cache keys
 
    log.debug("Clearing cache keys now...")
 
    CacheInvalidation.clear_cache()
 
    sa.commit()
 
    return added, removed
 

	
 

	
 
# set cache regions for beaker so celery can utilise it
 
def add_cache(settings):
 
    cache_settings = {'regions': None}
 
    for key in settings.keys():
 
        for prefix in ['beaker.cache.', 'cache.']:
 
            if key.startswith(prefix):
 
                name = key.split(prefix)[1].strip()
 
                cache_settings[name] = settings[key].strip()
 
    if cache_settings['regions']:
 
        for region in cache_settings['regions'].split(','):
 
            region = region.strip()
 
            region_settings = {}
 
            for key, value in cache_settings.items():
 
                if key.startswith(region):
 
                    region_settings[key.split('.')[1]] = value
 
            region_settings['expire'] = int(region_settings.get('expire',
 
                                                                60))
 
            region_settings.setdefault('lock_dir',
 
                                       cache_settings.get('lock_dir'))
 
            region_settings.setdefault('data_dir',
 
                                       cache_settings.get('data_dir'))
 

	
 
            if 'type' not in region_settings:
 
                region_settings['type'] = cache_settings.get('type',
 
                                                             'memory')
 
            beaker.cache.cache_regions[region] = region_settings
 

	
 

	
 
def load_rcextensions(root_path):
 
    import rhodecode
 
    from rhodecode.config import conf
 

	
 
    path = os.path.join(root_path, 'rcextensions', '__init__.py')
 
    if os.path.isfile(path):
 
        rcext = create_module('rc', path)
 
        EXT = rhodecode.EXTENSIONS = rcext
 
        log.debug('Found rcextensions now loading %s...' % rcext)
 

	
 
        # Additional mappings that are not present in the pygments lexers
 
        conf.LANGUAGES_EXTENSIONS_MAP.update(getattr(EXT, 'EXTRA_MAPPINGS', {}))
 

	
 
        #OVERRIDE OUR EXTENSIONS FROM RC-EXTENSIONS (if present)
 

	
 
        if getattr(EXT, 'INDEX_EXTENSIONS', []) != []:
 
            log.debug('settings custom INDEX_EXTENSIONS')
 
            conf.INDEX_EXTENSIONS = getattr(EXT, 'INDEX_EXTENSIONS', [])
 

	
 
        #ADDITIONAL MAPPINGS
 
        log.debug('adding extra into INDEX_EXTENSIONS')
 
        conf.INDEX_EXTENSIONS.extend(getattr(EXT, 'EXTRA_INDEX_EXTENSIONS', []))
 

	
 

	
 
#==============================================================================
 
# TEST FUNCTIONS AND CREATORS
 
#==============================================================================
 
def create_test_index(repo_location, config, full_index):
 
    """
 
    Makes default test index
 

	
 
    :param config: test config
 
    :param full_index:
 
    """
 

	
 
    from rhodecode.lib.indexers.daemon import WhooshIndexingDaemon
 
    from rhodecode.lib.pidlock import DaemonLock, LockHeld
 

	
 
    repo_location = repo_location
 

	
 
    index_location = os.path.join(config['app_conf']['index_dir'])
 
    if not os.path.exists(index_location):
 
        os.makedirs(index_location)
 

	
 
    try:
 
        l = DaemonLock(file_=jn(dn(index_location), 'make_index.lock'))
 
        WhooshIndexingDaemon(index_location=index_location,
 
                             repo_location=repo_location)\
 
@@ -603,27 +640,27 @@ class BasePasterCommand(Command):
 
                                       'actual_args': len(args)})
 

	
 
        # Decrement because we're going to lob off the first argument.
 
        # @@ This is hacky
 
        self.min_args -= 1
 
        self.bootstrap_config(args[0])
 
        self.update_parser()
 
        return super(BasePasterCommand, self).run(args[1:])
 

	
 
    def update_parser(self):
 
        """
 
        Abstract method.  Allows for the class's parser to be updated
 
        before the superclass's `run` method is called.  Necessary to
 
        allow options/arguments to be passed through to the underlying
 
        celery command.
 
        """
 
        raise NotImplementedError("Abstract Method.")
 

	
 
    def bootstrap_config(self, conf):
 
        """
 
        Loads the pylons configuration.
 
        """
 
        from pylons import config as pylonsconfig
 

	
 
        path_to_ini_file = os.path.realpath(conf)
 
        conf = paste.deploy.appconfig('config:' + path_to_ini_file)
 
        self.path_to_ini_file = os.path.realpath(conf)
 
        conf = paste.deploy.appconfig('config:' + self.path_to_ini_file)
 
        pylonsconfig.init_app(conf.global_conf, conf.local_conf)

Changeset was too big and was cut off... Show full diff anyway

0 comments (0 inline, 0 general)