Changeset - b66725ba01ed
[Not reviewed]
default
0 7 1
Thomas De Schampheleire - 7 years ago 2018-09-24 20:44:36
thomas.de_schampheleire@nokia.com
cli: add command 'kallithea-cli front-end-build'

Kallithea is under the GPL license, and we can thus only distribute any
generated code if we also ship the corresponding source.

We are moving towards a web front-end that use npm to download and compile
various open source components. The components might not be GPL, but if we
distribute any parts of their code (compiled or converted to other
representation), then we also must distribute the corresponding source under
the GPL.

It doesn't seem feasible for us to distribute the source of everything that
npm downloads and includes when we are building. It thus also doesn't seem
feasible for us to build and ship the compiled (possibly minified) front-end
code. Instead, we have to make it as smooth as possible for our users to
get up and running.

It doesn't seem feasible for us to ship or install npm. We must assume it is
available. That requirement must be documented clearly, and we must recommend
how to install npm for the most common platforms.

We could perhaps just document what manual steps to run. Kallithea doesn't
work out of the box anyway - it has to be configured and initialized. Extra
steps might not be a big problem.

Another approach is to call out to npm while pip is installing Kallithea and
download the requirements and build the files. It can be done by customizing
setuptools commands in setup.py. But: Python packaging is fragile. Even
though we only support pip, it really isn't built for things like this.
Custom output is muted and buffered and only shown if running with -v or the
command fails. And pip and setup.py can be used to build and install in so
many ways that we probably can't make it work reliably with all ways of
installing Kallithea.

The approach implemented by this commit is to add a custom cli command
'front-end-build' to run the required commands. This single user-facing
command can internally run various steps as needed. The only current
requirement is the presence of npm and an internet connection.

For now, this will just create/update style.css ... but currently probably
without any actual changes. The files created by npm (and the node_modules
directory) must *not* be a part of the release package made with 'setup.py
sdist'.

(Commit message is mostly written by Mads Kiilerich)
8 files changed with 64 insertions and 44 deletions:
0 comments (0 inline, 0 general)
docs/contributing.rst
Show inline comments
 
@@ -34,16 +34,15 @@ To get started with Kallithea developmen
 
        cd kallithea
 
        virtualenv ../kallithea-venv
 
        source ../kallithea-venv/bin/activate
 
        pip install --upgrade pip setuptools
 
        pip install --upgrade -e .
 
        pip install --upgrade -r dev_requirements.txt
 
        npm install     # install dependencies - both tools and data
 
        npm run less    # for generating css from less
 
        kallithea-cli config-create my.ini
 
        kallithea-cli db-create -c my.ini --user=user --email=user@example.com --password=password --repos=/tmp
 
        kallithea-cli front-end-build
 
        gearbox serve -c my.ini --reload &
 
        firefox http://127.0.0.1:5000/
 

	
 
If you plan to use Bitbucket_ for sending contributions, you can also fork
 
Kallithea on Bitbucket_ first (https://bitbucket.org/conservancy/kallithea) and
 
then replace the clone step above by a clone of your fork. In this case, please
docs/overview.rst
Show inline comments
 
@@ -66,17 +66,15 @@ installed.
 
  other package, usually as a ``.../site-packages/Kallithea-X-py2.7.egg/``
 
  directory with Python files and everything else that is needed.
 

	
 
  (``pip install kallithea`` from a source tree will do pretty much the same
 
  but build the Kallithea package itself locally instead of downloading it.)
 

	
 
.. note:: The front-end code is built with Node. Currently, it must be built
 
          locally after installing Kallithea. Assuming Node and the Node
 
          Package Manager is available, other tools and source code will be
 
          downloaded and installed. The front-end code can then be built from
 
          source locally.
 
.. note:: Kallithea includes front-end code that needs to be processed first.
 
The tool npm_ is used to download external dependencies and orchestrate the
 
processing. The ``npm`` binary must thus be available.
 

	
 

	
 
Web server
 
----------
 

	
 
Kallithea is (primarily) a WSGI_ application that must be run from a web
 
@@ -141,6 +139,7 @@ continuous hammering from the internet.
 
.. _nginx: http://nginx.org/en/
 
.. _iis: http://en.wikipedia.org/wiki/Internet_Information_Services
 
.. _pip: http://en.wikipedia.org/wiki/Pip_%28package_manager%29
 
.. _WSGI: http://en.wikipedia.org/wiki/Web_Server_Gateway_Interface
 
.. _HAProxy: http://www.haproxy.org/
 
.. _Varnish: https://www.varnish-cache.org/
 
.. _npm: https://www.npmjs.com/
docs/setup.rst
Show inline comments
 
@@ -2,28 +2,12 @@
 

	
 
=====
 
Setup
 
=====
 

	
 

	
 
Preparing front-end
 
-------------------
 

	
 
Temporarily, in the current Kallithea version, some extra steps are required to
 
build front-end files:
 

	
 
Find the right ``kallithea/public/less`` path with::
 

	
 
    python -c "import os, kallithea; print os.path.join(os.path.dirname(os.path.abspath(kallithea.__file__)), 'public', 'less')"
 

	
 
Then run::
 

	
 
    npm install
 
    npm run less
 

	
 

	
 
Setting up Kallithea
 
--------------------
 

	
 
First, you will need to create a Kallithea configuration file. Run the
 
following command to do so::
 

	
 
@@ -67,12 +51,16 @@ path to the root).
 
.. note:: the given path for Mercurial_ repositories **must** be write
 
          accessible for the application. It's very important since
 
          the Kallithea web interface will work without write access,
 
          but when trying to do a push it will fail with permission
 
          denied errors unless it has write access.
 

	
 
Finally, prepare the front-end by running::
 

	
 
    kallithea-cli front-end-build
 

	
 
You are now ready to use Kallithea. To run it simply execute::
 

	
 
    gearbox serve -c my.ini
 

	
 
- This command runs the Kallithea server. The web app should be available at
 
  http://127.0.0.1:5000. The IP address and port is configurable via the
docs/upgrade.rst
Show inline comments
 
@@ -83,24 +83,13 @@ If you originally installed using pip, i
 

	
 
If you originally installed from version control, it is as simple as::
 

	
 
    cd my-kallithea-clone
 
    hg pull -u
 
    pip install --upgrade -e .
 

	
 
Temporarily, in the current version, an extra step is required to build
 
front-end files:
 

	
 
Find the right ``kallithea/public/less`` path with::
 

	
 
    python -c "import os, kallithea; print os.path.join(os.path.dirname(os.path.abspath(kallithea.__file__)), 'public', 'less')"
 

	
 
Then run::
 

	
 
    npm install
 
    npm run less
 
    kallithea-cli front-end-build
 

	
 

	
 
5. Upgrade your configuration
 
-----------------------------
 

	
 
Run the following command to create a new configuration (``.ini``) file::
docs/usage/customization.rst
Show inline comments
 
@@ -25,19 +25,16 @@ Kallithea uses `Bootstrap 3`_ and Less_ 
 
to make some customizations, we recommend to do so by creating a ``theme.less``
 
file. When you create a file named ``theme.less`` in the Kallithea root
 
directory, you can use this file to override the default style. For example,
 
you can use this to override ``@kallithea-theme-main-color``,
 
``@kallithea-logo-url`` or other `Bootstrap variables`_.
 

	
 
After creating the ``theme.less`` file, you need to regenerate the CSS files.
 
Install npm for your platform and run::
 
After creating the ``theme.less`` file, you need to regenerate the CSS files, by
 
running::
 

	
 
    npm install
 
    npm run less
 

	
 
in the Kallithea root directory.
 
    kallithea-cli front-end-build --no-install-deps
 

	
 
.. _bootstrap 3: https://getbootstrap.com/docs/3.3/
 
.. _bootstrap variables: https://getbootstrap.com/docs/3.3/customize/#less-variables
 
.. _less: http://lesscss.org/
 

	
 

	
kallithea/bin/kallithea_cli.py
Show inline comments
 
@@ -17,10 +17,11 @@ from kallithea.bin.kallithea_cli_base im
 

	
 
# import commands (they will add themselves to the 'cli' object)
 
import kallithea.bin.kallithea_cli_celery
 
import kallithea.bin.kallithea_cli_config
 
import kallithea.bin.kallithea_cli_db
 
import kallithea.bin.kallithea_cli_extensions
 
import kallithea.bin.kallithea_cli_front_end
 
import kallithea.bin.kallithea_cli_iis
 
import kallithea.bin.kallithea_cli_index
 
import kallithea.bin.kallithea_cli_ishell
 
import kallithea.bin.kallithea_cli_repo
kallithea/bin/kallithea_cli_front_end.py
Show inline comments
 
new file 100644
 
# -*- coding: utf-8 -*-
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

	
 
import click
 
import kallithea.bin.kallithea_cli_base as cli_base
 

	
 
import os
 
import subprocess
 

	
 
import kallithea
 

	
 
@cli_base.register_command()
 
@click.option('--install-deps/--no-install-deps', default=True,
 
        help='Skip installation of dependencies, via "npm".')
 
@click.option('--generate/--no-generate', default=True,
 
        help='Skip generation of front-end files.')
 
def front_end_build(install_deps, generate):
 
    """Build the front-end.
 

	
 
    Install required dependencies for the front-end and generate the necessary
 
    files.  This step is complementary to any 'pip install' step which only
 
    covers Python dependencies.
 

	
 
    The installation of front-end dependencies happens via the tool 'npm' which
 
    is expected to be installed already.
 
    """
 
    rootdir = os.path.dirname(os.path.dirname(os.path.abspath(kallithea.__file__)))
 

	
 
    if install_deps:
 
        click.echo("Running 'npm install' to install front-end dependencies from package.json")
 
        subprocess.check_call(['npm', 'install'], cwd=rootdir)
 

	
 
    if generate:
 
        click.echo("Generating CSS")
 
        lesscpath = os.path.join(rootdir, 'node_modules', '.bin', 'lessc')
 
        lesspath = os.path.join(rootdir, 'kallithea', 'public', 'less', 'main.less')
 
        csspath = os.path.join(rootdir, 'kallithea', 'public', 'css', 'style.css')
 
        subprocess.check_call([lesscpath, '--relative-urls', '--source-map',
 
                '--source-map-less-inline', lesspath, csspath])
package.json
Show inline comments
 
@@ -4,11 +4,8 @@
 
  "dependencies": {
 
    "bootstrap": "3.3.7"
 
  },
 
  "devDependencies": {
 
    "less": "~2.7",
 
    "less-plugin-clean-css": "~1.5"
 
  },
 
  "scripts": {
 
    "less": "lessc --relative-urls --source-map --source-map-less-inline kallithea/public/less/main.less kallithea/public/css/style.css"
 
  }
 
}
0 comments (0 inline, 0 general)