Welcome to ksi-python’s documentation!

KSI Python SDK

This is a thin wrapper on top of KSI C SDK. Experimental, non-supported code.

Synopsis

Example of synchronous singing and verification.

import ksi
import hashlib

# Instantiate service parameters from the environment
KSI = ksi.KSI(**ksi.ksi_env())

# Sign a text string
sig = KSI.sign_hash(hashlib.sha256(b"Tere!"))
# Print some signature properties
print(sig.get_signing_time(), sig.get_signer_id())

# Now verify this text string, first obtaining a data hasher
h = sig.get_hasher()
h.update(b"Tere!")
print(KSI.verify_hash(sig, h))

# Obtain a binary blob which can be stored for long term
serialized_signature = sig.get_binary()

# Some time have passed, fetch the signature and verify again
sig2 = KSI.new_signature_object(serialized_signature)
print(KSI.verify_hash(sig2, h))

Note that asynchronous signing can provide a significant speedup when multiple signatures are requested. Example of asynchronous singing with gevent.

import ksi
import hashlib
from gevent.pool import Pool

# Instantiate service parameters from the environment
KSI = ksi.KSI(**ksi.ksi_env())

# Multiple strings to be signed
string_list = ["This", "is", "a", "list", "of", "strings"]

# Define a signer function.
def sign_hash(h):
  sig = KSI.sign_hash(h)
  # Verification and/or storing could be done here
  print(KSI.verify_hash(sig, h))

# Create a gevent pool. Note that for optimal efficiency
# pool size should not be smaller than
# ``KSI.get_async_config()['max_pending_count']``
pool = Pool(100)

# Sign all strings asynchronously
for string in string_list:
  pool.spawn(sign_hash, hashlib.sha256(string.encode()))
pool.join()

Client-side aggregation (signing in blocks) is also possible if the KSI Gateway allows it. This means that multiple hashes can be individually signed by a single request to the Gateway server. In addition, if block signing is allowed, the asynchronous signing service will by default dynamically sign in blocks depending on if the signing requests demand becomes too large to efficiently sign them one-by-one. Example of signing a block of hashes synchronously (asynchronous block signing is supported as well).

import ksi
import hashlib

# Instantiate service parameters from the environment
KSI = ksi.KSI(**ksi.ksi_env())

# Multiple strings to be signed
string_list = ["This", "is", "a", "list", "of", "strings"]

# Hashes of strings
hash_list = [hashlib.sha256(string.encode()) for string in string_list]

# Sign hashes in a block
sigs = KSI.sign_hash_list(hash_list)

# Verify hashes
for i in range(len(sigs)):
    print(KSI.verify_hash(sigs[i], hash_list[i]))

Install

  1. Requirements: Python 2.7+ or Python 3.1+. Jython, IronPython are not supported.

  2. Install fresh libksi aka KSI C SDK; see https://github.com/guardtime/libksi/

  3. Install python-devel package

  4. Run:

    > pip install ksi-python

or

> easy_install ksi-python

Tests

Specify KSI Gateway access parameters and run

> python setup.py test

To test if KSI Python SDK signs asynchronously with gevent, make sure gevent is installed.

Documentation

http://guardtime.github.io/ksi-python/

Type:

> pydoc ksi

to read the documentation after installation. Generating html or pdf documentation: make sure that dependencies like sphinx (pip install sphinx) are installed, extension is built (python setup.py build) and run:

> cd docs
> make html
or
> make latexpdf

License

Apache 2.0. Please contact Guardtime for supported options.

API Reference

class KSI

class ksi.KSI(signing_service, extending_service, publications_file='http://verify.guardtime.com/ksi-publications.bin', publications_cnstr={'email': 'publications@guardtime.com'})

KSI service provider class. Holds context and service parameters.

Parameters:
  • signing_service (dict) – Signing/aggregation service access parameters. A dictionary with following keys: {‘url’: .., ‘user’: …, ‘pass’: …}
  • extending_service (dict) – Extending service access parameters. A dictionary with following keys: {‘url’: .., ‘user’: …, ‘pass’: …}
  • publications_file (str, optional) – Publications file download URL. Default one works with Guardtime commercial KSI service.
  • publications_cnstr (dict, optional) –

    Publications file verification constraints: signing certificate (must be issued under a root in global system truststore) fields are validated against provided rules, e.g.:

    {"email" : "publications@guardtime.com"}

    Default one works with Guardtime commercial KSI service.
Properties:
verification: constants specifying verification policies. see set_verification_policy()
extend(sig)

Extend the signature to the latest available publication.

Modifies the signature in-place. Extender sever must be specified earlier. Possible, if there is at least 1 publication created after signing, i.e. quite safe to try if 35 days have passed from signing.

Extending a signature is very useful before long-term archiving. This makes distant-future verification possible without access to extender service.

Parameters:KsiSignature –
Returns:True if extending was successful, False if no suitable publication was found.
Raises:ksi.Error – or its subclass on KSI errors
get_async_config()

Get the current async configuration.

The maximum aggregator request count (from KSI.get_async_config) is the maximum number of requests the server is willing to respond to per round. Setting a max_req_count significantly larger than that will most probably cause failures and the async service to unnecessarily resend often failing requests in cache.

See set_async_config()

Returns:{“max_req_count”: (int), “max_pending_count”: (int), “max_aggregator_req_count”: (int)}
Return type:dict
get_async_state()

Get the async state.

See KsiAsync.get_state()

get_verification_policy()

Retrieves the current verification policy.

Returns:Identifier. see set_verification_policy() for a list of values.
Return type:int
new_signature_object(blob)

Instantiates a KsiSignature object from earlier serialized binary representation.

Parameters:str/buffer – binary representation of signature data.
Returns:A newly instantiated KsiSignature object.
Return type:KsiSignature
Raises:ksi.Error – or its subclass on KSI errors
set_async_config(config)

Set the async configuration.

Note max_pending_count cannot be assigned a smaller value than previously set. Setting it significantly higher will cause request timeouts (meaning a high rate of request resending) and possibly AsyncServiceError instances.

Note that if max_pending_count is reached then new requests will be cached in a queue until the number of pending requests becomes smaller. The user is responsible for memory safety of the queued requests, possibly using gevent.pool with a max size of max_pending_count + queue max size (depending on the amount of memory the program should have access to).

Note

The asynchronous service maximum request count is initially capped at 250 responses per round (one round lasts for 1 second) and maximum pending count at 500. Even if the maximum pending count is allowed to be set higher, network errors may occur, as the host may not be able to send/receive all requests on time, depending on the host’s NIC, CPU, etc.

Parameters:

dict – {‘max_req_count’: (int), ‘max_pending_count’: (int)}
Raises:
  • ValueError – when input contains invalid values or if ‘max_pending_count’ is smaller than previously set
  • TypeError – when input contains invalid keys
set_verification_policy(new_policy)

Specify a verification policy for future verifications.

Note that it is hard to fail with the default one. Change only if necessary. Suffix _EXT instructs verification to transparently extend the signature if possible.
Parameters:policy_id –

A property from the KSI object:

KSI.verification.POLICY_GENERAL
KSI.verification.POLICY_GENERAL_EXT
KSI.verification.POLICY_KEY_BASED
KSI.verification.POLICY_CALENDAR_BASED
KSI.verification.POLICY_PUBLICATIONS_FILE_BASED
KSI.verification.POLICY_PUBLICATIONS_FILE_BASED_EXT
sign_blob(blob)

Signs a data blob.

Note

In case of gevent use (function run by spawned greenlet):
Signing is an asynchronous operation.
Otherwise:
Signing blocks for 1..2 seconds.
Parameters:str/buffer – data to be signed.
Returns:see KsiSignature class in this module.
Return type:KsiSignature
Raises:ksi.Error – or its subclass on KSI errors
sign_hash(data_hash)

Signs a data hash.

Note

In case of gevent use (function run by spawned greenlet):
Signing is an asynchronous operation.
Otherwise:
Signing blocks for 1..2 seconds.

When using gevent, the state of the async service can be monitored by calling KSI.get_async_state(). Spawning greenlets does not automatically guarantee that that the async state is updated as greenlets may not run instantaneously. It is recommended to also monitor how many greenlets are being spawned or even use gevent.pool in accordance with the async service configuration (see KSI.get_async_config()) for increased memory safety.

Parameters:

Object – hash created using a hasher from Python hashlib.
Returns:

see KsiSignature class in this module.
Return type:

KsiSignature
Raises:
  • ksi.Error – or its subclass on KSI errors
  • ksi.AsyncServiceError – when the request has waited over a minute to be sent out or when the maximum retry count has been reached due to previous errors. Related to too many asynchronous requests or incorrect async service config
sign_hash_list(data_hashes)

Signs a list of hashes in one block.

Note

In case of gevent use (function run by spawned greenlet):
Signing is an asynchronous operation.
Otherwise:
Signing blocks for 1..2 seconds.
Parameters:

list – hashes created using a hasher from Python hashlib.
Returns:

list of KsiSignature instances
Return type:

[KsiSignature]
Raises:
verify(sig)

Verify signature consistency. No document content check.

No return value.

Raises:ksi.Error – or its subclass on KSI errors
verify_blob(sig, blob)

Verify a data blob and its signature.

Parameters:
  • KsiSignature – Signature object.
  • blob – A string or buffer containing binary data bytes.
Returns:

A 3-tuple with following data:

True if signature is OK; False if there is conslusive evidence that signature is invalid and further action can not change the situation. Code: error code as string. See https://github.com/guardtime/libksi/blob/v3.16.2482/src/ksi/policy.h#L70. Reason: Human readable message.
Return type:

(result (boolean), code (str), reason (str))
Raises:
  • ValueError – Final decision is not possible given current input data. Changing something, e.g. verification policy, might help to find a conclusive answer.
  • ksi.Error – or its subclass on KSI errors
verify_hash(sig, data_hash)

Verify a signature and a provided data hash of signed data.

Parameters:
  • KsiSignature – Signature object.
  • data_hash – A hasher from hashlib, after processing its input data.
Returns:

A 3-tuple with following data:

True if signature is OK; False if there is conslusive evidence that signature is invalid and further action can not change the situation. Code: error code as string. See https://github.com/guardtime/libksi/blob/v3.16.2482/src/ksi/policy.h#L70. Reason: Human readable message.
Return type:

(result (boolean), code (str), reason (str))
Raises:
  • ValueError – Final decision is not possible given current input data. Changing something, e.g. verification policy, might help to find a conclusive answer.
  • ksi.Error – or its subclass on KSI errors

class KsiSignature

class ksi.KsiSignature(ksi, sig)

KSI Signature object.

Exists only in instantiated, integrity-checked form. Created by the KSI class only. Do not initialize directly.

Parameters:
  • ksi (class KSI) – KSI class. Necessary to avoid early garbage collection.
  • sig – serialized signature.
get_binary()

Returns a serialized signature, ready to be passed to C language binding or stored into a file or db field.

get_data_hash()

Returns the hash algorithm and hash value of signed data

See the note about get_hash_algorithm()

Returns:A tuple of hash algorithm name and a binary data hash.
Return type:(str, str/buffer)
Raises:ksi.Error – or its subclass on KSI errors
get_hash_algorithm()

Returns the hash algorithm name used for hashing data during signing.

It is important to use exactly the same hash algorithm for hashing data during signing and during signature verification.

Note

Returns algorithm names like SHA2-256. Some older libraries, like Python’s hashlib and OpenSSL use names like SHA256. Use some processing to get legacy compatible name, for example:

sig.get_hash_algorithm().replace(“SHA2-“, “sha”)
Returns:hash algorithm name
Return type:str
Raises:ksi.Error – or its subclass on KSI errors
get_hasher()

Returns a data hasher object from hashlib.

Used hash algorithm matches the one which was used during the signature creation.

Returns:

a new hashing object from hashlib
Return type:

Object
Raises:
  • ksi.Error – or its subclass on KSI errors
  • ValueError – hashlib does not support used hash algorithm
get_publication_data()

Returns publication data used for extending this signature.

Returned data can be used to validate the signature in strongest possible form, using an independent publication. Publications references refer to the choice of mediums; publication code can be compared with one printed in chosen medium.

Returns:None if publication record is not available. A dictionary on success, see example below:
{
    'refs': [
        u'ref1',
        u'Financial Times, ISSN: 0307-1766, 2015-06-17',
        u'https://twitter.com/Guardtime/status/611103980870070272'
    ],
    'publication': 'AAAAAA-CVPYKY-AANVCV-Q7IJFM-ZJ5YBK-BNXRKG-IWIFER-5XXL73-4NXXLS-D6CROT-QUMAHA-JDWBQI',
    'publishing_time_t': 1434326400L,
    'publishing_time': datetime.datetime(2015, 6, 15, 0, 0)
}
Raises:ksi.Error – or its subclass on KSI errors
get_signer_id()

Extract cryptographically protected data signer’s identity.

This identifier contains hierarchical namespace of KSI aggregators, gateway and end user. May look like GT :: GT :: Company :: user.name.

Returns:a signer’s id prefixed with full aggregator hierarchy.
Return type:str
Raises:ksi.Error – or its subclass on KSI errors
get_signing_time()

Extract cryptographically protected signing time from a signature.

Returns:datetime.datetime object, in local timezone.
Return type:Object
Raises:ksi.Error – or its subclass on KSI errors
get_signing_time_utc()

Extract cryptographically protected signing time from a signature.

Returns:datetime.datetime object, using UTC timezone.
Return type:Object
Raises:ksi.Error – or its subclass on KSI errors
is_extended()

Checks if signature is extened and contains a publication record.

Returns:True if extended, False if not.
Return type:Bool
Raises:ksi.Error – or its subclass on KSI errors
set_binary(sig)

Sets a new serialized signature. Called after successfully extending the prior signature. Internal use.

class KsiAsync

class ksi.KsiAsync(ctx, surl, suser, skey)

Async service class. Handles the KSI asynchronous signing service.

Created by the KSI class only. Do not initialize or call methods directly.

Note that this class is optimized to be used with gevent. Requests called without spawned greenlets (e.g. by gevent.spawn or gevent.Pool.spawn) will be blocking.

get_config()

Returns async service configuration

Note

Mutable entries (see KsiAsync.set_config):
‘max_req_count’, ‘max_pending_count’, ‘dynamic_block_signing’
Immutable entries:
‘server_max_req_count’:
maximum number of requests the aggregator is willing to respond to per round
‘max_block_size’:
capped at 2^16, max block size gives the maximum number of hashes one block can contain
‘block_signing’:
if block signing is available for the current ksi context. If not, ‘dynamic_block_signing’ cannot be set to True and KSI.sign_hash_list raises KSI.AsyncServiceError
Returns:
{“server_max_req_count”: int, “max_req_count”: int,
”max_pending_count”: int, “max_block_size”: int, “dynamic_block_signing”: bool, “block_signing”: bool}
Return type:dict
get_response(request_id)

Returns the serialized signature

Removes request and response data from instance and returns the received signature.

Parameters:uuid – id of request
Returns:signature of the hash with given uuid
Return type:sig
get_state()

Returns async service state

state dictionary key/value pairs:
“pending requests”:
(int) number of requests which have been sent for signing. Cannot be larger than KsiAsync.max_pending_count
“responses ready”:
(int) number of signatures which have been received but not returned by KSI.sign_hash
“waiting in queue”:
(int) number of signatures which have not yet been sent for signing as KsiAsync.max_pending_count has been reached (if a request stays in queue for more than 1 minute, then AsyncServiceError is raised)
“requests in cache”:
(int) number of active KSI.sign_hash and KSI.sign_hash_list instances
Returns:dict (state)
is_pending(request_id)

Checks if request hash has been signed

Parameters:uuid – id of request
Returns:True if requested hash has been signed
Return type:bool
new_block_request(hash_names, hash_digests, leaf_ids=None)

Creates block signing request and sends to the service.

Parameters:
  • [string] – list of hashing function names
  • [Object] – list of hashes created by hashlib
  • [uuid] – list of hash uuids if block signing request is dynamically initialized by KsiAsync
Returns:

id of the request so its progress can be tracked
Return type:

uuid
Raises:
new_request(hash_name, hash_digest)

Creates request and sends to the service.

Parameters:
  • string – name of the hashing function which created this hash.
  • Object – hash created using a hasher from Python hashlib
Returns:

id of the request so its progress can be tracked
Return type:

uuid
Raises:

ksi.Error – or its subclass on KSI errors
run_service(request_id)

Runs the async signing service

_ksi.ksi_run_async_service returns:
dict/bool:
{uuid: signature} in case the async response handle returned a signature response
int:
is the number of responses yet to be received
list/bool:
[response_id, error_string] if the async response handle returned an error
Parameters:

uuid – in order to track current request
Raises:
set_config(max_req_count=None, max_pending_count=None, dynamic_block_signing=None)

Saves new async service configuration

Parameters:
  • int – maximum request count per round, should not be larger than the server maximum request count per round, as timeouts and HTTP errors will occur
  • int – maximum number of requests to be cached at one time, must not be smaller than last configured value
  • bool – sign with blocks explicitly or in case demand exceeds max_pending_count, not permitted to use if config['block_signing'] == False
Raises:

ValueError – when parameters are incorrect

Helpers

ksi.ksi_env()

Helper for initializing a KSI class with KSI service parameters from the environment.

System environment parameters should look like:
Required:
KSI_AGGREGATOR=’url=http://url user=name pass=xyz’ KSI_EXTENDER=’url=http://url user=name pass=xyz’
Required if using Guardtime commerical KSI service:
KSI_PUBFILE=’http://verify.guardtime.com/ksi-publications.bin’ KSI_PUBFILE_CNSTR=’{“email” : “publications@guardtime.com”}’

Example

KSI = ksi.KSI(**ksi.ksi_env())

Raises:ksi.FormatError – when an environment variable misses required component.

Exceptions

exception ksi.AsyncServiceError

Bases: ksi.Error

  • Request max retry count reached.
  • HTTP error (possibly caused by requesting significantly more responses than the aggregator can handle, see KSI.set_async_config()).
  • Network error (possibly caused by sening too many requests and receiving timeout errors, see KSI.set_async_config()).
  • Async service request cache full.
  • Async connection was closed.
  • Async service has not finished.
  • Request timeout (possibly caused by too many pending requests or initializing a synchronous sign requests on top of a full request queue).
  • If KSI.sign_hash_list is called, but block signing is not allowed.
exception ksi.ConfigurationError

Bases: ksi.Error

Necessary parameters are not configured

exception ksi.CryptoError

Bases: ksi.Error

  • Cryptographic operation could not be performed. Likely causes are unsupported cryptographic algorithms, invalid keys and lack of resources.
exception ksi.Error

Bases: exceptions.Exception

Unspecfied KSI error

exception ksi.FormatError

Bases: ksi.Error

Invalid argument, format, or parameter. Untrusted or unavailable hash algorithm.

exception ksi.HashError

Bases: ksi.Error

  • Unknown hash algorithm.
  • Untrusted hash algorithm.
exception ksi.InvalidSignatureError

Bases: ksi.Error

  • Invalid KSI signature.
  • Invalid PKI signature.
  • The PKI signature is not trusted by the API.
  • The objects used are in an invalid state.
exception ksi.NetworkError

Bases: ksi.Error

  • A network error occurred.
  • A network connection timeout occurred.
  • A network send timeout occurred.
  • A network receive timeout occurred.
  • A HTTP error occurred.
exception ksi.PublicationError

Bases: ksi.Error

Problems with publication based verification.

  • The extender returned a wrong calendar chain.
  • No suitable publication to extend to.
  • The publication in the signature was not found in the publications file.
  • Invalid publication.
  • The publications file is not signed.
exception ksi.ServiceAuthError

Bases: ksi.Error

  • The request could not be authenticated (missing or unknown login identifier, MAC check failure, etc).
exception ksi.ServiceError

Bases: ksi.Error

Various KSI service related errors.

  • The request is still pending.
  • The request ID in response does not match with request ID in request.
  • Pattern for errors with client request.
  • The request contained invalid payload (unknown payload type, missing mandatory elements, unknown critical elements, etc).
  • The server encountered an unspecified internal error.
  • The server encountered unspecified critical errors connecting to upstream servers.
  • No response from upstream aggregators.
  • The extender returned an error.
  • The request indicated client-side aggregation tree larger than allowed for the client (retrying would not succeed either).
  • The request combined with other requests from the same client in the same round would create an aggregation sub-tree larger than allowed for the client (retrying in a later round could succeed).
  • Too many requests from the client in the same round (retrying in a later round could succeed)
  • Input hash value in the client request is longer than the server allows.
  • Received PDU v2 response to PDU v1 request. Configure the SDK to use PDU v2 format for the given aggregator.
  • Received PDU v1 response to PDU v2 request. Configure the SDK to use PDU v1 format for the given aggregator.
exception ksi.ServiceExtenderBlockchainError

Bases: ksi.ServiceError

The Extender server does not have required period of Calendar Blockchain.

  • The request asked for a hash chain going backwards in time
  • Pattern for local errors in the server.
  • The server misses the internal database needed to service the request (most likely it has not been initialized yet).
  • The server’s internal database is in an inconsistent state.
  • The request asked for hash values older than the oldest round in the server’s database.
  • The request asked for hash values newer than the newest round in the server’s database.