Changeset - a60cd29ba7e2
[Not reviewed]
default
0 6 0
Marcin Kuzminski - 15 years ago 2010-10-10 21:34:24
marcin@python-works.com
more docs update
6 files changed with 272 insertions and 131 deletions:
0 comments (0 inline, 0 general)
README.rst
Show inline comments
 
------------------------------------------------
 
Pylons based repository management for mercurial
 
------------------------------------------------
 

	
 
RhodeCode (RhodiumCode)
 
=======================
 

	
 
``RhodeCode`` (formerly hg-app) is Pylons based repository management and 
 
serving for mercurial_. It's similar to github or bitbucket, but it's suppose to run
 
as standalone app, it's open source and focuses more on restricted access to repositories
 
There's no default free access to RhodeCode You have to create an account in order
 
to use the application. It's powered by vcs_ library that we created to handle
 
many various version control systems.
 

	
 
RhodeCode uses `Semantic Versioning <http://semver.org/>`_
 

	
 

	
 
RhodeCode demo
 
--------------
 

	
 
http://hg.python-works.com
 

	
 
The default access is
 

	
 
**Overview**
 
- username: demo
 
- password: demo
 

	
 
Source code
 
-----------
 

	
 
Source code is along with issue tracker is available at
 
http://bitbucket.org/marcinkuzminski/rhodecode
 

	
 
- Has it's own middleware to handle mercurial protocol request. Each request can 
 
Also a source codes can be obtained from demo rhodecode instance
 
http://hg.python-works.com/rhodecode/summary
 

	
 
Instalation
 
-----------
 

	
 
 Please visit http://packages.python.org/rhodecode/installation.html
 

	
 

	
 
Features
 
--------
 

	
 
- Has it's own middleware to handle mercurial_ protocol request. Each request can 
 
  be logged and authenticated. Runs on threads unlikely to hgweb You can make
 
  multiple pulls/pushes simultaneous
 
- Full permissions and authentication per project private/read/write/admin. 
 
  One account for web interface and mercurial push/pull/clone.
 
  One account for web interface and mercurial_ push/pull/clone.
 
- Mako templates let's you customize look and feel of application.
 
- Beautiful diffs, annotations and source codes all colored by pygments.
 
- Mercurial branch graph and yui-flot powered graphs with zooming and statistics
 
- Mercurial_ branch graph and yui-flot powered graphs with zooming and statistics
 
- Admin interface with user/permission management. User activity journal logs
 
  pulls, pushes, forks,registrations. Possible to disable built in hooks
 
- Server side forks, it's possible to fork a project and hack it free without
 
  breaking the main.   
 
- Full text search on source codes, search on file names. All powered by whoosh
 
  and build in indexing daemons
 
  (no external search servers required all in one application)
 
- Rss / atom feeds, gravatar support, download sources as zip/tarballs  
 
- Async tasks for speed and performance using celery (works without them too)  
 
- Async tasks for speed and performance using celery_ (works without them too)  
 
- Backup scripts can do backup of whole app and send it over scp to desired 
 
  location
 
- Setup project descriptions and info inside built in db for easy, non 
 
  file-system operations
 
- Added cache with invalidation on push/repo management for high performance and
 
  always up to date data. 
 
- Based on pylons 1.0 / sqlalchemy 0.6 / sqlite
 

	
 
**Incoming**
 

	
 
Incoming
 
--------
 

	
 
- code review based on hg-review (when it's stable)
 
- git support (when vcs can handle it - almost there !)
 
- commit based wikis
 
- clonning from remote repositories into rhodecode (git/mercurial)
 
- code review (probably based on hg-review)
 
- git_ support (when vcs_ can handle it - it's almost there !)
 
- commit based build in wiki system
 
- clone points and cloning from remote repositories into rhodecode (git_ and mercurial_)
 
- some cache optimizations
 
- other cools stuff that i can figure out (or You can help me figure out)
 
   
 
------------
 
Installation
 
------------
 

	
 
**quick setup**
 
License
 
-------
 
 
 
- pip install -E rhodecode-venv rhodecode
 
- activate virtualenv
 
- run `paster make-config RhodeCode production.ini`
 
- run `paster setup-app production.ini`
 
- run `paster runserver production.ini`
 
``rhodecode`` is released under GPL_ license.
 

	
 
You're ready to go.
 

	
 
**MORE DETAILED INSTRUCTIONS**
 

	
 
- I highly recommend to install new virtualenv for rhodecode see 
 
  http://pypi.python.org/pypi/virtualenv for more details.
 
- Create new virtualenv using `virtualenv --no-site-packages /var/www/rhodecode-venv`
 
  this will install new virtual env into /var/www/rhodecode-venv. 
 
  Activate the virtualenv by running 
 
  `source activate /var/www/rhodecode-venv/bin/activate`   
 
- Make a folder for rhodecode somewhere on the filesystem for example /var/www/rhodecode  
 
- Run easy_install rhodecode
 
- Run `paster make-config RhodeCode production.inii` in order to install 
 
  the application config. You can play with the app settings later 
 
- Run `paster setup-app production.ini` it should create all needed tables 
 
  and an admin account make sure You specify correct path to repositories. 
 
- Remember that the given path for mercurial repositories must be write 
 
  accessible for the application
 
- Run paster serve production.ini - or you can use sample init.d scripts.
 
  the app should be available at the 127.0.0.1:5000
 
- Use admin account you created to login.
 
- Default permissions on each repository is read, and owner is admin. So remember
 
  to update these.
 
- In order to use full power of async tasks, You must install message broker
 
  preferably rabbitmq and start celeryd daemon together with rhodecode. 
 
  The app should gain a lot of speed and become much more responsible. 
 
  For installation instructions You can visit: 
 
  http://ask.github.com/celery/getting-started/index.html. 
 
- All needed configs are inside rhodecode sources ie. celeryconfig.py, 
 
  development.ini, production.ini You can configure the email, ports, loggers, 
 
  workers from there.
 
- For full text search You can either put crontab entry for 
 
  `python /var/www/rhodecode/rhodecode/lib/indexers/daemon.py incremental <path_to_repos>`
 
  or run indexer from admin panel. This will scann the repos given in the 
 
  application setup or given path for daemon.py and each scann in incremental 
 
  mode will scann only changed files.
 
\ No newline at end of file
 
Documentation
 
-------------
 

	
 
 Online documentation for current version is available at
 
 http://packages.python.org/rhodecode/.
 
 You may also build documentation for yourself - go into ``docs/`` and run::
 

	
 
   make html
 

	
docs/conf.py
Show inline comments
 
@@ -117,13 +117,13 @@ html_theme_path = ['theme']
 
# pixels large.
 
#html_favicon = None
 

	
 
# Add any paths that contain custom static files (such as style sheets) here,
 
# relative to this directory. They are copied after the builtin static files,
 
# so a file named "default.css" will overwrite the builtin "default.css".
 
html_static_path = ['_static']
 
#html_static_path = ['_static']
 

	
 
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
 
# using the given strftime format.
 
#html_last_updated_fmt = '%b %d, %Y'
 

	
 
# If true, SmartyPants will be used to convert quotes and dashes to
docs/index.rst
Show inline comments
 
.. _index:
 

	
 
Welcome to RhodeCode (RhodiumCode) documentation!
 
=================================================
 

	
 
``RhodeCode`` is Pylons based repository management and serving for mercurial. 
 
It's similar to github or bitbucket, but focuses more on closed access restrictions.
 
``RhodeCode`` (formerly hg-app) is Pylons based repository management and 
 
serving for mercurial_. It's similar to github or bitbucket, but it's suppose to run
 
as standalone app, it's open source and focuses more on restricted access to repositories
 
There's no default free access to RhodeCode You have to create an account in order
 
to use the application. 
 
to use the application. It's powered by vcs_ library that we created to handle
 
many various version control systems.
 

	
 
**RhodeCode demo is available at**
 
RhodeCode uses `Semantic Versioning <http://semver.org/>`_
 

	
 

	
 
RhodeCode demo
 
--------------
 

	
 
http://hg.python-works.com
 

	
 
The default access is
 

	
 
- username: demo
 
- password: demo
 

	
 
RhodeCode uses `Semantic Versioning <http://semver.org/>`_
 
Source code
 
-----------
 

	
 
Source code is along with issue tracker is available at
 
http://bitbucket.org/marcinkuzminski/rhodecode
 

	
 
**Features**
 
Also a source codes can be obtained from demo rhodecode instance
 
http://hg.python-works.com/rhodecode/summary
 

	
 
- Has it's own middleware to handle mercurial protocol request. Each request can 
 
Features
 
--------
 

	
 
- Has it's own middleware to handle mercurial_ protocol request. Each request can 
 
  be logged and authenticated. Runs on threads unlikely to hgweb You can make
 
  multiple pulls/pushes simultaneous
 
- Full permissions and authentication per project private/read/write/admin. 
 
  One account for web interface and mercurial push/pull/clone.
 
  One account for web interface and mercurial_ push/pull/clone.
 
- Mako templates let's you customize look and feel of application.
 
- Beautiful diffs, annotations and source codes all colored by pygments.
 
- Mercurial branch graph and yui-flot powered graphs with zooming and statistics
 
- Mercurial_ branch graph and yui-flot powered graphs with zooming and statistics
 
- Admin interface with user/permission management. User activity journal logs
 
  pulls, pushes, forks,registrations. Possible to disable built in hooks
 
- Server side forks, it's possible to fork a project and hack it free without
 
  breaking the main.   
 
- Full text search on source codes, search on file names. All powered by whoosh
 
  and build in indexing daemons
 
  (no external search servers required all in one application)
 
- Rss / atom feeds, gravatar support, download sources as zip/tarballs  
 
- Async tasks for speed and performance using celery (works without them too)  
 
- Async tasks for speed and performance using celery_ (works without them too)  
 
- Backup scripts can do backup of whole app and send it over scp to desired 
 
  location
 
- Setup project descriptions and info inside built in db for easy, non 
 
  file-system operations
 
- Added cache with invalidation on push/repo management for high performance and
 
  always up to date data. 
 
@@ -55,36 +69,49 @@ RhodeCode uses `Semantic Versioning <htt
 
.. figure::  images/screenshot2_summary_page.png
 
   :align:   left
 

	
 
   Summary page
 

	
 

	
 
**Incoming**
 
Incoming
 
--------
 

	
 
- code review based on hg-review (when it's stable)
 
- git support (when vcs can handle it - almost there !)
 
- commit based wikis
 
- clonning from remote repositories into rhodecode (git/mercurial)
 
- code review (probably based on hg-review)
 
- git_ support (when vcs_ can handle it - it's almost there !)
 
- commit based build in wiki system
 
- clone points and cloning from remote repositories into rhodecode (git_ and mercurial_)
 
- some cache optimizations
 
- other cools stuff that i can figure out (or You can help me figure out)
 

	
 
License
 
-------
 

	
 
``rhodecode`` is released under GPL_ license.
 

	
 

	
 
Documentation
 
=============
 
-------------
 

	
 
**Installation:**
 

	
 
.. toctree::
 
   :maxdepth: 1
 

	
 
   installation
 
   setup
 

	
 
Other topics
 
============
 
------------
 

	
 
* :ref:`genindex`
 
* :ref:`search`
 

	
 
.. _virtualenv: http://pypi.python.org/pypi/virtualenv
 
.. _python: http://www.python.org/
 
.. _django: http://www.djangoproject.com/
 
.. _mercurial: http://mercurial.selenic.com/
 
.. _subversion: http://subversion.tigris.org/
 
.. _git: http://git-scm.com/
 
\ No newline at end of file
 
.. _git: http://git-scm.com/
 
.. _celery: http://celeryproject.org/
 
.. _Sphinx: http://sphinx.pocoo.org/
 
.. _GPL: http://www.gnu.org/licenses/gpl.html
 
.. _vcs: http://pypi.python.org/pypi/vcs
docs/installation.rst
Show inline comments
 
@@ -3,18 +3,16 @@
 
Installation
 
============
 

	
 
``RhodeCode`` is written entirely in Python, but in order to use it's full
 
potential there are some third-party requirements. When RhodeCode is used 
 
together with celery You have to install some kind of message broker,
 
recommended one is rabbitmq to make the async tasks work.
 
For installation instructions You can visit: 
 
http://ask.github.com/celery/getting-started/index.html.
 
recommended one is rabbitmq_ to make the async tasks work.
 

	
 
Of course RhodeCode works in sync mode also, then You don't have to install
 
any third party apps. Celery will give You large speed improvement when using
 
any third party apps. Celery_ will give You large speed improvement when using
 
many big repositories. If You plan to use it for 2 or 3 small repositories, it
 
will work just fine without celery running.
 
   
 
After You decide to Run it with celery make sure You run celeryd and
 
message broker together with the application.   
 

	
 
@@ -23,12 +21,15 @@ Requirements for Celery
 

	
 
**Message Broker** 
 

	
 
- preferred is `RabbitMq <http://www.rabbitmq.com/>`_
 
- possible other is `Redis <http://code.google.com/p/redis/>`_
 

	
 
For installation instructions You can visit: 
 
http://ask.github.com/celery/getting-started/index.html
 
It's very nice tutorial how to start celery_ with rabbitmq_
 

	
 
Install from Cheese Shop
 
------------------------
 

	
 
Easiest way to install ``rhodecode`` is to run::
 

	
 
@@ -41,55 +42,48 @@ Or::
 
If you prefer to install manually simply grab latest release from
 
http://pypi.python.org/pypi/rhodecode, decompres archive and run::
 

	
 
   python setup.py install
 

	
 

	
 
**Setting up the application**
 
I recommend to run the RhodeCode in separate virtualenv.
 
See http://pypi.python.org/pypi/virtualenv for more details.
 

	
 
- run `paster make-config RhodeCode production.ini` make specific application
 
  config, 
 
- run `paster setup-app production.ini` makes the database, and propagates it
 
  with default data, In this step You have to provide admin username and repositories
 
  location, it can be a new location or with existing ones in that case RhodeCode
 
  will scann all new found repos and put it into database.
 
- run `paster runserver production.ini` runs the server.
 

	
 

	
 
**STEP BY STEP EXAMPLE INSTRUCTION**
 
Step by step installation example
 
---------------------------------
 

	
 

	
 
- Assuming You have setup virtualenv create one using 
 
  `virtualenv --no-site-packages /var/www/rhodecode-venv`
 
  this will install new virtual env into /var/www/rhodecode-venv. 
 
- Activate the virtualenv by running 
 
  `source activate /var/www/rhodecode-venv/bin/activate`   
 
- Assuming You have installed virtualenv_ create one using. The `--no-site-packages`
 
  will make sure non of Your system libs are linked with this virtualenv_  
 

	
 
::
 

	
 
 virtualenv --no-site-packages /var/www/rhodecode-venv
 

	
 
- this will install new virtualenv_ into `/var/www/rhodecode-venv`. 
 
- Activate the virtualenv_ by running 
 

	
 
::
 

	
 
  source activate /var/www/rhodecode-venv/bin/activate
 
     
 
- Make a folder for rhodecode somewhere on the filesystem for example 
 
  /var/www/rhodecode  
 
- Run easy_install rhodecode, this will install rhodecode together with pylons
 

	
 
::
 

	
 
  mkdir /var/www/rhodecode
 
  
 
    
 
- Run this command to install rhodecode
 

	
 
::
 

	
 
  easy_install rhodecode 
 

	
 
- this will install rhodecode together with pylons
 
  and all other required python libraries
 
- Run `paster make-config RhodeCode production.ini` in order to install 
 
  the application config. 
 
- Run `paster setup-app production.ini` it should create all needed tables 
 
  and an admin account. Also make sure You specify correct path to repositories.
 
  You can either use a new location of one with already exising ones. RhodeCode
 
  will simply add all new found repositories to it's database. 
 
- Remember that the given path for mercurial repositories must be write 
 
  accessible for the application. It's very important since RhodeCode web interface
 
  will work even without such an access but, when trying to do a push it's eventually
 
  failed with permission denied. 
 
- Run `paster serve production.ini`
 
  the app should be available at the 127.0.0.1:5000
 
- Use admin account you created to login.
 
- Default permissions on each repository is read, and owner is admin. So remember
 
  to update these.
 

	
 

	
 
You can now proceed to :ref:`setup`
 

	
 
- All needed configs are inside rhodecode sources ie. celeryconfig.py, 
 
  development.ini, production.ini You can configure the email, ports, loggers, 
 
  workers from there.
 
- For full text search You can either put crontab entry for 
 
  `python /var/www/rhodecode/rhodecode/lib/indexers/daemon.py incremental <path_to_repos>`
 
  or run indexer from admin panel. This will scann the repos given in the 
 
  application setup or given path for daemon.py and each scann in incremental 
 
  mode will scan only changed files.
 
\ No newline at end of file
 
.. _virtualenv: http://pypi.python.org/pypi/virtualenv  
 
.. _python: http://www.python.org/
 
.. _mercurial: http://mercurial.selenic.com/
 
.. _celery: http://celeryproject.org/
 
.. _rabbitmq: http://www.rabbitmq.com/
 
\ No newline at end of file
docs/setup.rst
Show inline comments
 
.. _setup:
 

	
 
Setup
 
=====
 

	
 

	
 
Setting up the application
 
--------------------------
 

	
 
::
 
 
 
 paster make-config RhodeCode production.ini
 

	
 
- This will create `production.ini` config inside the directory
 
  this config contain various settings for rhodecode, e.g port, email settings
 
  static files, cache and logging.
 

	
 
::
 

	
 
 paster setup-app production.ini` 
 

	
 
- This command will create all needed tables and an admin account. 
 
  When asked for a path You can either use a new location of one with already 
 
  existing ones. RhodeCode will simply add all new found repositories to 
 
  it's database. Also make sure You specify correct path to repositories.
 
- Remember that the given path for mercurial_ repositories must be write 
 
  accessible for the application. It's very important since RhodeCode web interface
 
  will work even without such an access but, when trying to do a push it'll 
 
  eventually faile with permission denied errors. 
 
- Run 
 

	
 
::
 
 
 
 paster serve production.ini
 
 
 
- This command runs the rhodecode server the app should be available at the 
 
  127.0.0.1:5000. This ip and port is configurable via the production.ini 
 
  file  created in previos step
 
- Use admin account you created to login.
 
- Default permissions on each repository is read, and owner is admin. So 
 
  remember to update these.
 

	
 
- All needed configs are inside rhodecode sources ie. celeryconfig.py, 
 
  development.ini, production.ini You can configure the email, ports, loggers, 
 
  workers from there.
 
  
 
Setting up Whoosh
 
-----------------
 

	
 
- For full text search You can either put crontab entry for 
 
  `python /var/www/rhodecode/rhodecode/lib/indexers/daemon.py incremental <path_to_repos>`
 
  or run indexer from admin panel. This will scann the repos given in the 
 
  application setup or given path for daemon.py and each scann in incremental 
 
  mode will scan only changed files.
 

	
 
::
 
 
 
 python /var/www/rhodecode/rhodecode/lib/indexers/daemon.py incremental <put_here_path_to_repos>
 
  
 
When using incremental mode whoosh will check last modification date of each file
 
and add it to reindex if newer file is available. Also indexing daemon checks
 
for removed files and removes them from index. Sometime You might want to rebuild
 
index from scrach, in admin pannel You can check `build from scratch` flag
 
and in standalone daemon You can pass `full` instead on incremental to build
 
remove previos index and build new one.
 

	
 
Nginx virtual host example
 
--------------------------
 

	
 
Sample config for nginx::
 

	
 
 server {
 
    listen          80;
 
    server_name     hg.myserver.com;
 
    access_log      /var/log/nginx/rhodecode.access.log;
 
    error_log      /var/log/nginx/rhodecode.error.log;
 
    location / {
 
            root /var/www/rhodecode/rhodecode/public/;
 
            if (!-f $request_filename){
 
                proxy_pass      http://127.0.0.1:5000;
 
            }
 
            #this is important for https !!!
 
            proxy_set_header X-Url-Scheme $scheme;
 
            include         /etc/nginx/proxy.conf;  
 
    }
 
 }  
 
  
 
Here's the proxy.conf. It's tunned so it'll not timeout on long
 
pushes and also on large pushes::
 
  
 
TODO: write that !
 
\ No newline at end of file
 
    proxy_redirect              off;
 
    proxy_set_header            Host $host;
 
    proxy_set_header            X-Host $http_host;
 
    proxy_set_header            X-Real-IP $remote_addr;
 
    proxy_set_header            X-Forwarded-For $proxy_add_x_forwarded_for;
 
    proxy_set_header            Proxy-host $proxy_host;
 
    client_max_body_size        400m;
 
    client_body_buffer_size     128k;
 
    proxy_buffering             off;
 
    proxy_connect_timeout       3600;
 
    proxy_send_timeout          3600;
 
    proxy_read_timeout          3600;
 
    proxy_buffer_size           8k;
 
    proxy_buffers               8 32k;
 
    proxy_busy_buffers_size     64k;
 
    proxy_temp_file_write_size  64k;
 
 
 
Also when using root path with nginx You might set the static files to false
 
in production.ini file::
 

	
 
  [app:main]
 
    use = egg:rhodecode
 
    full_stack = true
 
    static_files = false
 
    lang=en
 
    cache_dir = %(here)s/data
 

	
 
To not have the statics served by the application.
 

	
 

	
 
.. _virtualenv: http://pypi.python.org/pypi/virtualenv
 
.. _python: http://www.python.org/
 
.. _mercurial: http://mercurial.selenic.com/
 
.. _celery: http://celeryproject.org/
 
.. _rabbitmq: http://www.rabbitmq.com/
 
\ No newline at end of file
setup.py
Show inline comments
 
@@ -12,12 +12,20 @@ requirements = [
 
        "whoosh==1.0.0",
 
        "celery>=2.0.0",
 
        "py-bcrypt",
 
        "babel",
 
    ]
 

	
 
classifiers = ['Development Status :: 4 - Beta',
 
                   'Environment :: Web Environment',
 
                   'Framework :: Pylons',
 
                   'Intended Audience :: Developers',
 
                   'License :: OSI Approved :: BSD License',
 
                   'Operating System :: OS Independent',
 
                   'Programming Language :: Python', ]
 

	
 
if sys.version_info < (2, 6):
 
    requirements.append("simplejson")
 
    requirements.append("pysqlite")
 

	
 
#additional files from project that goes somewhere in the filesystem
 
#relative to sys.prefix
 
@@ -48,18 +56,19 @@ packages = find_packages(exclude=['ez_se
 

	
 
setup(
 
    name='RhodeCode',
 
    version=get_version(),
 
    description=description,
 
    long_description=long_description,
 
    keywords='mercurial web hgwebdir replacement serving hgweb rhodecode',
 
    keywords='rhodiumcode mercurial web hgwebdir replacement serving hgweb rhodecode',
 
    license='BSD',
 
    author='Marcin Kuzminski',
 
    author_email='marcin@python-works.com',
 
    url='http://hg.python-works.com',
 
    install_requires=requirements,
 
    classifiers=classifiers,
 
    setup_requires=["PasteScript>=1.6.3"],
 
    data_files=data_files,
 
    packages=packages,
 
    include_package_data=True,
 
    test_suite='nose.collector',
 
    package_data=package_data,
0 comments (0 inline, 0 general)