Changeset - 053983a464e4
[Not reviewed]
beta
0 5 0
Marcin Kuzminski - 15 years ago 2011-02-17 00:44:45
marcin@python-works.com
docs and readme update
5 files changed with 82 insertions and 28 deletions:
0 comments (0 inline, 0 general)
README.rst
Show inline comments
 

	
 
=================================================
 
Welcome to RhodeCode (RhodiumCode) documentation!
 
=================================================
 

	
 
``RhodeCode`` (formerly hg-app) is Pylons framework based Mercurial repository 
 
browser/management with build in push/pull server and full text search.
 
It works on http/https, has build in permission/authentication(+ldap) features 
 
It's similar to github or bitbucket, but it's suppose to run as standalone 
 
hosted application, it's open source and focuses more on restricted access to 
 
repositories. It's powered by vcs_ library that me and Lukasz Balcerzak created 
 
to handle many various version control systems.
 
browser/management tool with build in push/pull server and full text search.
 
It works on http/https, has build in permission/authentication system with 
 
ability to auth via LDAP. It's similar in some parts to github or bitbucket, 
 
but it's suppose to run as standalone hosted application, it's open source 
 
and donation ware and focuses more on providing customized, self administered 
 
interface for Mercurial(and soon GIT) repositories. It's powered by vcs_ 
 
library that me and Lukasz Balcerzak created to handle many various version 
 
control systems.
 

	
 
RhodeCode uses `Semantic Versioning <http://semver.org/>`_
 

	
 
RhodeCode demo
 
--------------
 

	
 
http://hg.python-works.com
 

	
 
The default access is anonymous but You can login to administrative account
 
using those credentials
 

	
 
- username: demo
 
- password: demo
 

	
 
Source code
 
-----------
 

	
 
The most up to date sources can be obtained from my own RhodeCode instance
 
https://rhodecode.org 
 

	
 
Rarely updated source code and issue tracker is available at bitbcuket
 
http://bitbucket.org/marcinkuzminski/rhodecode
 

	
 
Installation
 
------------
 

	
 
 Please visit http://packages.python.org/RhodeCode/installation.html
 

	
 

	
 
Features
 
--------
 

	
 
- Has it's own middleware to handle mercurial_ protocol request. 
 
  Each request can be logged and authenticated. Runs on threads unlikely to 
 
  hgweb. You can make multiple pulls/pushes simultaneous. Supports http/https 
 
  and ldap
 
  and LDAP
 
- Full permissions (private/read/write/admin) and authentication per project. 
 
  One account for web interface and mercurial_ push/pull/clone operations.
 
- Have built in users groups for easier permission management
 
- Users can fork other users repo. RhodeCode have also compare view to see
 
  combined changeset for all changeset made within single push.
 
- Mako templates let's you customize look and feel of application.
 
- Beautiful diffs, annotations and source codes all colored by pygments.
 
- Beautiful diffs, annotations and source codes all colored by pygments. Raw
 
  diffs are made in git-diff format, including git binary-patches
 
- Mercurial_ branch graph and yui-flot powered graphs with zooming and statistics
 
- Admin interface with user/permission management. Admin activity journal, logs
 
  pulls, pushes, forks, registrations and other actions made by all users.
 
- Server side forks, it's possible to fork a project and hack it free without
 
  breaking the main repository.
 
  breaking the main repository. You can even write Your own hooks and install 
 
  them
 
- Full text search powered by Whoosh on source codes, and file names.
 
  Build in indexing daemons, with optional incremental index build
 
  (no external search servers required all in one application)
 
  (no external search servers required. All in one application !)
 
- Setup project descriptions and info inside built in db for easy, non 
 
  file-system operations
 
- Intelligent cache with invalidation after push or project change, provides high 
 
  performance and always up to date data.    
 
- Intelligent cache with invalidation after push or project change, provides 
 
  high performance and always up to date data.    
 
- Rss / atom feeds, gravatar support, download sources as zip/tar/gz
 
- Async tasks for speed and performance using celery_ (works without them too)  
 
- Backup scripts can do backup of whole app and send it over scp to desired 
 
  location 
 
- Based on pylons / sqlalchemy / sqlite / whoosh / vcs
 

	
 

	
 
.. include:: ./docs/screenshots.rst
 
    
 
    
 
Incoming / Plans
 
----------------
 

	
 
- project grouping
 
- User groups/teams
 
- code review (probably based on hg-review)
 
- full git_ support, with push/pull server (currently in beta tests)
 
- redmine integration
 
- public accessible activity feeds
 
- commit based build in wiki system
 
- clone points and cloning from remote repositories into rhodecode 
 
  (git_ and mercurial_)
 
- more statistics and graph (global annotation + some more statistics)
 
- other cools stuff that i can figure out (or You can help me figure out)
 

	
 
License
 
-------
 

	
 
``rhodecode`` is released under GPL_ license.
 

	
 

	
 
Mailing group Q&A
 
-----------------
 

	
 
join the `Google group <http://groups.google.com/group/rhodecode>`_
 

	
 
open an issue at `issue tracker <http://bitbucket.org/marcinkuzminski/rhodecode/issues>`_
 

	
 
join #rhodecode on FreeNode (irc.freenode.net)
docs/contributing.rst
Show inline comments
 
.. _contributing:
 

	
 
Contributing in RhodeCode
 
=========================
 

	
 
If You would like to contribute to RhodeCode, please contact me, any help is
 
greatly appreciated.
 

	
 
Preferable method Would be to fork RhodeCode repository from bitbucket
 
https://bitbucket.org/marcinkuzminski/rhodecode and then open a pull request.
 
This way it's easier for me to merge.
 

	
 
To run RhodeCode in a development version You always need to install tip
 
version of RhodeCode and VCS library.
 

	
 

	
 
Thank You.
docs/installation.rst
Show inline comments
 
.. _installation:
 

	
 
Installation
 
============
 

	
 
``RhodeCode`` is written entirely in Python, but in order to use it's full
 
potential there are some third-party requirements. When RhodeCode is used 
 
together with celery You have to install some kind of message broker,
 
recommended one is rabbitmq_ to make the async tasks work.
 

	
 
Of course RhodeCode works in sync mode also, then You don't have to install
 
any third party apps. Celery_ will give You large speed improvement when using
 
many big repositories. If You plan to use it for 7 or 10 small repositories, it
 
will work just fine without celery running.
 
   
 
After You decide to Run it with celery make sure You run celeryd using paster
 
and message broker together with the application.   
 

	
 
Install from Cheese Shop
 
------------------------
 
Rhodecode requires python 2.x greater than version 2.5
 

	
 
Easiest way to install ``rhodecode`` is to run::
 

	
 
 easy_install rhodecode
 
    easy_install rhodecode
 

	
 
Or::
 

	
 
 pip install rhodecode
 
    pip install rhodecode
 

	
 
If you prefer to install manually simply grab latest release from
 
http://pypi.python.org/pypi/rhodecode, decompres archive and run::
 

	
 
   python setup.py install
 
    python setup.py install
 

	
 

	
 
Step by step installation example
 
---------------------------------
 

	
 

	
 
- Assuming You have installed virtualenv_ create one using. 
 
  The `--no-site-packages` will make sure non of Your system libs are linked 
 
  with this virtualenv_  
 

	
 
::
 

	
 
 virtualenv --no-site-packages /var/www/rhodecode-venv
 
    virtualenv --no-site-packages /var/www/rhodecode-venv
 

	
 
- this will install new virtualenv_ into `/var/www/rhodecode-venv`. 
 
- Activate the virtualenv_ by running 
 

	
 
::
 

	
 
  source /var/www/rhodecode-venv/bin/activate
 
    source /var/www/rhodecode-venv/bin/activate
 
     
 
- Make a folder for rhodecode somewhere on the filesystem for example 
 

	
 
::
 

	
 
  mkdir /var/www/rhodecode
 
    mkdir /var/www/rhodecode
 
  
 
    
 
- Run this command to install rhodecode
 

	
 
::
 

	
 
  easy_install rhodecode 
 
    easy_install rhodecode 
 

	
 
- this will install rhodecode together with pylons
 
  and all other required python libraries
 

	
 
Requirements for Celery (optional)
 
----------------------------------
 

	
 
.. note::
 
   Installing message broker and using celery is optional, RhodeCode will
 
   work without them perfectly fine.
 

	
 

	
 
**Message Broker** 
 

	
 
- preferred is `RabbitMq <http://www.rabbitmq.com/>`_
 
- possible other is `Redis <http://code.google.com/p/redis/>`_
 

	
 
For installation instructions You can visit: 
 
http://ask.github.com/celery/getting-started/index.html
 
It's very nice tutorial how to start celery_ with rabbitmq_
 

	
 

	
 
You can now proceed to :ref:`setup`
 
-----------------------------------
docs/setup.rst
Show inline comments
 
.. _setup:
 

	
 
Setup
 
=====
 

	
 

	
 
Setting up the application
 
--------------------------
 

	
 
First You'll ned to create RhodeCode config file. Run the following command 
 
First You'll need to create RhodeCode config file. Run the following command 
 
to do this
 

	
 
::
 
 
 
 paster make-config RhodeCode production.ini
 

	
 
- This will create `production.ini` config inside the directory
 
  this config contains various settings for RhodeCode, e.g proxy port, 
 
  email settings, usage of static files, cache, celery settings and logging.
 

	
 

	
 
Next we need to create the database. I'll recommend to use sqlite (default) 
 
or postgresql. Make sure You properly adjust the db url in the .ini file to use
 
other than the default sqlite database
 

	
 
Next we need to create the database.
 

	
 
::
 

	
 
 paster setup-app production.ini
 

	
 
- This command will create all needed tables and an admin account. 
 
  When asked for a path You can either use a new location of one with already 
 
  existing ones. RhodeCode will simply add all new found repositories to 
 
  it's database. Also make sure You specify correct path to repositories.
 
- Remember that the given path for mercurial_ repositories must be write 
 
  accessible for the application. It's very important since RhodeCode web 
 
  interface will work even without such an access but, when trying to do a 
 
  push it'll eventually fail with permission denied errors. 
 

	
 
You are ready to use rhodecode, to run it simply execute
 
You are ready to use RhodeCode, to run it simply execute
 

	
 
::
 
 
 
 paster serve production.ini
 
 
 
- This command runs the RhodeCode server the app should be available at the 
 
  127.0.0.1:5000. This ip and port is configurable via the production.ini 
 
  file created in previous step
 
- Use admin account you created to login.
 
- Default permissions on each repository is read, and owner is admin. So 
 
  remember to update these if needed. In the admin panel You can toggle ldap,
 
  anonymous, permissions settings. As well as edit more advanced options on 
 
  users and repositories
 
  
 
Using RhodeCode with SSH
 
------------------------
 

	
 
RhodeCode repository structures are kept in directories with the same name 
 
as the project, when using repository groups, each group is a a subdirectory.
 
This will allow You to use ssh for accessing repositories quite easy. There
 
are some exceptions when using ssh for accessing repositories.
 

	
 
You have to make sure that the webserver as well as the ssh users have unix
 
permission for directories. Secondly when using ssh rhodecode will not 
 
@@ -71,55 +73,55 @@ if Your main directory (the same as set 
 
for example `\home\hg` and repository You are using is `rhodecode`
 

	
 
The command runned should look like this::
 
 hg clone ssh://user@server.com/home/hg/rhodecode
 
 
 
Using external tools such as mercurial server or using ssh key based auth is
 
fully supported.
 
    
 
Setting up Whoosh full text search
 
----------------------------------
 

	
 
Starting from version 1.1 whoosh index can be build using paster command.
 
You have to specify the config file that stores location of index, and
 
location of repositories (`--repo-location`). Starting from version 1.2 it is 
 
also possible to specify a comma separated list of repositories (`--index-only`)
 
to build index only on chooses repositories skipping any other found in repos
 
location
 

	
 
There is possible also to pass `-f` to the options
 
to enable full index rebuild. Without that indexing will run always in in
 
incremental mode.
 

	
 
incremental mode::
 

	
 
 paster make-index production.ini --repo-location=<location for repos> 
 
	paster make-index production.ini --repo-location=<location for repos> 
 

	
 

	
 

	
 
for full index rebuild You can use::
 

	
 
 paster make-index production.ini -f --repo-location=<location for repos>
 
	paster make-index production.ini -f --repo-location=<location for repos>
 

	
 

	
 
building index just for chosen repositories is possible with such command::
 
 
 
 paster make-index production.ini --repo-location=<location for repos> --index-only=vcs,rhodecode
 

	
 

	
 
In order to do periodical index builds and keep Your index always up to date.
 
It's recommended to do a crontab entry for incremental indexing. 
 
An example entry might look like this
 

	
 
::
 
 
 
 /path/to/python/bin/paster /path/to/rhodecode/production.ini --repo-location=<location for repos> 
 
  
 
When using incremental (default) mode whoosh will check last modification date 
 
of each file and add it to reindex if newer file is available. Also indexing 
 
daemon checks for removed files and removes them from index. 
 

	
 
Sometime You might want to rebuild index from scratch. You can do that using 
 
the `-f` flag passed to paster command or, in admin panel You can check 
 
`build from scratch` flag.
 

	
 

	
 
@@ -313,52 +315,60 @@ Directory ::
 
 Login Attribute      = sAMAccountName
 
 First Name Attribute = givenName
 
 Last Name Attribute  = sn
 
 E-mail Attribute     = mail
 

	
 
All other LDAP settings will likely be site-specific and should be
 
appropriately configured.
 

	
 
Setting Up Celery
 
-----------------
 

	
 
Since version 1.1 celery is configured by the rhodecode ini configuration files
 
simply set use_celery=true in the ini file then add / change the configuration 
 
variables inside the ini file.
 

	
 
Remember that the ini files uses format with '.' not with '_' like celery
 
so for example setting `BROKER_HOST` in celery means setting `broker.host` in
 
the config file.
 

	
 
In order to make start using celery run::
 

	
 
 paster celeryd <configfile.ini>
 

	
 

	
 

	
 
.. note::
 
   Make sure You run this command from same virtualenv, and with the same user
 
   that rhodecode runs.
 
   
 
HTTPS support
 
-------------
 

	
 
There are two ways to enable https, first is to set HTTP_X_URL_SCHEME in
 
Your http server headers, than rhodecode will recognise this headers and make
 
proper https redirections, another way is to set `force_https = true` 
 
in the ini cofiguration to force using https, no headers are needed than to
 
enable https
 

	
 

	
 
Nginx virtual host example
 
--------------------------
 

	
 
Sample config for nginx using proxy::
 

	
 
 server {
 
    listen          80;
 
    server_name     hg.myserver.com;
 
    access_log      /var/log/nginx/rhodecode.access.log;
 
    error_log       /var/log/nginx/rhodecode.error.log;
 
    location / {
 
            root /var/www/rhodecode/rhodecode/public/;
 
            if (!-f $request_filename){
 
                proxy_pass      http://127.0.0.1:5000;
 
            }
 
            #this is important if You want to use https !!!
 
            proxy_set_header X-Url-Scheme $scheme;
 
            include         /etc/nginx/proxy.conf;  
 
    }
 
 }  
 
  
 
Here's the proxy.conf. It's tuned so it'll not timeout on long
 
@@ -405,68 +415,93 @@ Sample config for apache using proxy::
 
    
 
            <Proxy *>
 
              Order allow,deny
 
              Allow from all
 
            </Proxy>
 
    
 
            #important !
 
            #Directive to properly generate url (clone url) for pylons
 
            ProxyPreserveHost On
 
    
 
            #rhodecode instance
 
            ProxyPass / http://127.0.0.1:5000/
 
            ProxyPassReverse / http://127.0.0.1:5000/
 
            
 
            #to enable https use line below
 
            #SetEnvIf X-Url-Scheme https HTTPS=1
 
            
 
    </VirtualHost> 
 

	
 

	
 
Additional tutorial
 
http://wiki.pylonshq.com/display/pylonscookbook/Apache+as+a+reverse+proxy+for+Pylons
 

	
 

	
 
Apache as subdirectory
 
----------------------
 

	
 

	
 
Apache subdirectory part::
 

	
 
    <Location /rhodecode>
 
      ProxyPass http://127.0.0.1:59542/rhodecode
 
      ProxyPassReverse http://127.0.0.1:59542/rhodecode
 
      SetEnvIf X-Url-Scheme https HTTPS=1
 
    </Location> 
 

	
 
Besides the regular apache setup You'll need to add such part to .ini file::
 

	
 
    filter-with = proxy-prefix
 

	
 
Add the following at the end of the .ini file::
 

	
 
    [filter:proxy-prefix]
 
    use = egg:PasteDeploy#prefix
 
    prefix = /<someprefix> 
 

	
 

	
 
Apache's example FCGI config
 
----------------------------
 

	
 
TODO !
 

	
 
Other configuration files
 
-------------------------
 

	
 
Some example init.d script can be found here, for debian and gentoo:
 

	
 
https://rhodeocode.org/rhodecode/files/tip/init.d
 

	
 

	
 
Troubleshooting
 
---------------
 

	
 
- missing static files ?
 

	
 
 - make sure either to set the `static_files = true` in the .ini file or
 
   double check the root path for Your http setup. It should point to 
 
   for example:
 
   /home/my-virtual-python/lib/python2.6/site-packages/rhodecode/public
 
   
 
- can't install celery/rabbitmq
 

	
 
 - don't worry RhodeCode works without them too. No extra setup required
 

	
 
- long lasting push timeouts ?
 

	
 
 - make sure You set a longer timeouts in Your proxy/fcgi settings, timeouts
 
   are caused by https server and not RhodeCode
 

	
 
- large pushes timeouts ?
 
 
 
 - make sure You set a proper max_body_size for the http server
 

	
 
- Apache doesn't pass basicAuth on pull/push ?
 

	
 
 - Make sure You added `WSGIPassAuthorization true` 
 

	
 
.. _virtualenv: http://pypi.python.org/pypi/virtualenv
 
.. _python: http://www.python.org/
 
.. _mercurial: http://mercurial.selenic.com/
 
.. _celery: http://celeryproject.org/
 
.. _rabbitmq: http://www.rabbitmq.com/
 
.. _python-ldap: http://www.python-ldap.org/
docs/upgrade.rst
Show inline comments
 
@@ -4,48 +4,53 @@ Upgrade
 
=======
 

	
 
Upgrade from Cheese Shop
 
------------------------
 

	
 
Easiest way to upgrade ``rhodecode`` is to run::
 

	
 
 easy_install -U rhodecode
 

	
 
Or::
 

	
 
 pip install --upgrade rhodecode
 

	
 

	
 
Then make sure You run from the installation directory
 

	
 
::
 
 
 
 paster make-config RhodeCode production.ini
 
 
 
This will display any changes made from new version of RhodeCode To your
 
current config. And tries to do an automerge. It's always better to do a backup
 
of config file and recheck the content after merge.
 

	
 
.. note::
 
   The next steps only apply to upgrading from non bugfix releases eg. from
 
   1.1 to 1.2. Bugfix releases (eg. 1.1.2->1.1.3) will not have any database 
 
   schema changes or whoosh library updates
 

	
 
It's also good to rebuild the whoosh index since after upgrading the whoosh 
 
version there could be introduced incompatible index changes.
 
version there could be introduced incompatible index changes. 
 

	
 

	
 
The last step is to upgrade the database. To do this simply run
 

	
 
::
 

	
 
 paster upgrade-db production.ini
 
    paster upgrade-db production.ini
 
 
 
This will upgrade schema, as well as update some default on the database,
 
always recheck the settings of the application, if there are no new options
 
that need to be set.
 

	
 
.. note::
 
   Always perform a database backup before doing upgrade.
 

	
 

	
 

	
 
.. _virtualenv: http://pypi.python.org/pypi/virtualenv  
 
.. _python: http://www.python.org/
 
.. _mercurial: http://mercurial.selenic.com/
 
.. _celery: http://celeryproject.org/
 
.. _rabbitmq: http://www.rabbitmq.com/
 
\ No newline at end of file
0 comments (0 inline, 0 general)