kallithea Files · scripts/validate-commits

Files @ 3c503044e9f1

Branch filter:

Location: kallithea/scripts/validate-commits

3c503044e9f1 1.7 KiB text/plain Show Annotation Show as Raw Download as Raw

mads

mysql: bump sqlalchemy.url MariaDB/MySQL charset to to 'utf8mb4' to get full UTF-8 support

The change in 210e76d69b62 only changed character_set_database, as shown by output after:

--- a/kallithea/model/base.py
+++ b/kallithea/model/base.py
@@ -46,3 +46,8 @@ def init_model(engine):
engine_str = obfuscate_url_pw(str(engine.url))
log.info("initializing db for %s", engine_str)
meta.Base.metadata.bind = engine
+
+ meta.Session.configure(bind=engine)
+ for a, b in meta.Session().execute('''show variables''').fetchall():
+ if 'character_set_' in a:
+ print(a, repr(b))

Before, with charset=utf8, the utf8mb3 charset was used all the way through the stack:

[kallithea.model.base] initializing db for mysql://kallithea-test:XXXXX@localhost/kallithea-test?charset=utf8
character_set_client 'utf8'
character_set_connection 'utf8'
character_set_database 'utf8mb4'
character_set_filesystem 'binary'
character_set_results 'utf8'
character_set_server 'latin1'
character_set_system 'utf8'

With explicit charset=utf8mb4:

[kallithea.model.base] initializing db for mysql://kallithea-test:XXXXX@localhost/kallithea-test?charset=utf8mb4
character_set_client 'utf8mb4'
character_set_connection 'utf8mb4'
character_set_database 'utf8mb4'
character_set_filesystem 'binary'
character_set_results 'utf8mb4'
character_set_server 'latin1'
character_set_system 'utf8'

#!/bin/bash
# Validate the specified commits against test suite and other checks.

if [ -n "$VIRTUAL_ENV" ]; then
    echo "Please run this script from outside a virtualenv."
    exit 1
fi

if ! hg update --check -q .; then
    echo "Working dir is not clean, please commit/revert changes first."
    exit 1
fi

revset=$1
if [ -z "$revset" ]; then
    echo "Warning: no revisions specified, checking draft changes up to the current one."
    revset='draft() and ancestors(.)'
fi

venv=$(mktemp -d kallithea-validatecommits-env-XXXXXX)
resultfile=$(mktemp kallithea-validatecommits-result-XXXXXX)
echo > "$resultfile"

cleanup()
{
    rm -rf /tmp/kallithea-test*
    rm -rf "$venv"
}
finish()
{
    cleanup
    # print (possibly intermediate) results
    cat "$resultfile"
    rm "$resultfile"
}
trap finish EXIT

for rev in $(hg log -r "$revset" -T '{node}\n'); do
    hg log -r "$rev"
    hg update "$rev"

    cleanup
    python3 -m venv "$venv"
    source "$venv/bin/activate"
    pip install --upgrade pip setuptools
    pip install -e . -r dev_requirements.txt python-ldap python-pam

    # run-all-cleanup
    if ! scripts/run-all-cleanup ; then
        echo "run-all-cleanup encountered errors!"
        result="NOK"
    else
        if ! hg update --check -q .; then
            echo "run-all-cleanup did not give clean results!"
            result="NOK"
            hg diff
            hg revert -a
        else
            result=" OK"
        fi
    fi
    echo "$result: $rev (run-all-cleanup)" >> "$resultfile"

    # pytest
    if py.test; then
        result=" OK"
    else
        result="NOK"
    fi
    echo "$result: $rev (pytest)" >> "$resultfile"

    deactivate
    echo
done