Development

Contribute

Please, fork the code on Github and develop your feature in a new branch split from the develop branch. Commit your code to the main project by sending a pull request onto the develop branch

Setup

# Clone project
git clone git@github.com:DLR-SC/prov-db-connector.git
cd prov-db-connector

# Setup virtual environment
virtualenv -p /usr/bin/python3.4 env
source env/bin/activate

# Install dependencies
make dev-setup

Execute tests

make test

Coverage report

make coverage

Compile documentation

make docs

Create new database adapters

The database adapters are the binding class to the actual database. If you are consider to build your own adapter please keep in mind:

  • All adapters must enhance the Baseadapter class.
  • You must implement all specified functions in BaseAdapter
  • You should test it via the AdapterTestTemplate class template.
  • You should test it also via the ProvDbTestTemplate class template.

1. - Create your database adapter

First you must create a class that extend from Baseadapter and implement all functions.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
import logging
from uuid import uuid4

from prov.constants import PROV_ASSOCIATION, PROV_TYPE, PROV_MENTION
from provdbconnector.db_adapters.baseadapter import BaseAdapter, DbRecord, DbRelation, METADATA_KEY_IDENTIFIER, \
    METADATA_KEY_PROV_TYPE
from provdbconnector.exceptions.database import InvalidOptionsException, NotFoundException
from provdbconnector.utils.serializer import encode_dict_values_to_primitive, split_into_formal_and_other_attributes, \
    merge_record

log = logging.getLogger(__name__)


class SimpleInMemoryAdapter(BaseAdapter):
    """
    The simple in memory adapter is a reference implementation for a database adapter to save prov information
    into a graph database


    For exmaple to use the simple db_adapter use the following script

    .. literalinclude:: ../examples/simple_example.py
            :linenos:
            :language: python

    """
    all_nodes = dict()  # separate dict for records only (to get them by id)
    """
    Contains all nodes
    """

2. - Create test suites

To test your adapter you should create two test suits:

See this example tests for the SimpleInMemoryAdapter

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
from provdbconnector.exceptions.database import InvalidOptionsException
from provdbconnector.db_adapters.in_memory import SimpleInMemoryAdapter
from provdbconnector.prov_db import ProvDb
from provdbconnector.tests import AdapterTestTemplate
from provdbconnector.tests import ProvDbTestTemplate


class SimpleInMemoryAdapterTest(AdapterTestTemplate):
    """
    This class implements the AdapterTestTemplate and only override some functions.

    """
    def setUp(self):
        """
        Connect to your database

        """
        self.instance = SimpleInMemoryAdapter()
        self.instance.connect(None)

    def test_connect_invalid_options(self):
        """
        Test your connect function with invalid data

        """
        auth_info = {"invalid": "Invalid"}
        with self.assertRaises(InvalidOptionsException):
            self.instance.connect(auth_info)

    def clear_database(self):
        """
        Clear the database

        """
        self.instance.all_nodes = dict()
        self.instance.all_relations= dict()

    def tearDown(self):
        """
        Delete your instance

        """
        del self.instance


class SimpleInMemoryAdapterProvDbTests(ProvDbTestTemplate):
    """
    This is the high level test for the SimpleInMemoryAdapter

    """
    def setUp(self):
        """
        Setup a ProvDb instance
        """
        self.provapi = ProvDb(api_id=1, adapter=SimpleInMemoryAdapter, auth_info=None)

    def clear_database(self):
        """
        Clear function get called before each test starts

        """
        self.provapi._adapter.all_nodes = dict()
        self.provapi._adapter.all_relations = dict()

    def tearDown(self):
        """
        Delete prov api instance
        """
        del self.provapi

3. - Implement your adapter logic

The last step is to create your logic inside the SimpleInMemoryAdapter for example the save_record and get_record functions:

Now you are ready to implement all other functions.

Note

If you don’t know where should you start Start with the first test and try to implement functions successively according to the tests and look into the documentation of the AdapterTestTemplate

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
    """
    Contains all relation according to the following structure
    `(start_identifier, (end_identifier,attributes, metadata))``
    """

    def __init__(self, *args):
        """
        Init the adapter without any params
        :param args:
        """
        super(SimpleInMemoryAdapter, self).__init__()
        pass

    def connect(self, authentication_info):
        """
        This function setups your database connection (auth / service discover)

        :param authentication_info: The info to connect to the db
        :type authentication_info: dict or None
        :return: The result of the connection attempt
        :rtype: Bool
        """

        if authentication_info is not None:
            raise InvalidOptionsException()
        :type to_node: prov.model.Identifier
        :param attributes: The actual provenance data
        :type attributes: dict
        :param metadata: Some metadata that are not PROV-O related
        :type metadata: dict
        :return: The id of the relation
        :rtype: str
        """

        # save all relation information and return the relation id as string