In this article, we're going to talk about how you can make new handler integrations into the MindsDB codebase.
MindsDB is a cloud database service with a lot of AI and data integrations that help you design, train, and predict over the cloud!
MindsDB's community has always been welcoming all contributions over their repositories. Since they have such nice support for their products alongside their active community, I decided to write down this quick tutorial on how you can contribute to MindsDB by creating your handler integrations.
This guide is created for those who want to contribute to
mindsdb/mindsdb so without further ado, let's talk about the steps.
Fork & Clone
First thing first, make sure that you have
mindsdb/mindsdb forked and cloned on your machine. Change your directory to where the codebase is on.
Create A Handler
I've written a tool that helps you create MindsDB handlers with a few question answerings. It's basically a Cookiecutter template that provides whatever you may need during your handler development. Follow the steps to create a basic handler template for yourself.
1. Install Cookiecutter
Make sure you have
pip installed and updated on your machine and install the
cookiecutter package via the following command.
pip install -U cookiecutter
To test it out and find out whether the installation was successful, run it with the
2. Generate the template
Now, it's time to use my tool called
lnxpy/cookiecutter-mindsdb to generate a simple handler for yourself. MindsDB stores all the handlers in
mindsdb/integrations/handlers/ so we need to put our handler in that path too. Keep in mind that we've already changed our directory to
cookiecutter gh:lnxpy/cookiecutter-mindsdb -o mindsdb/integrations/handlers/
After running the above command, it tries to retrieve the content of the cookiecutter template and then, it asks you a bunch of questions in order to create the most suitable handler for you. The following table helps you answer the questions accurately.
|Handler name||Name of your handler|
|Handler slug||Slug version of your handler name|
|Description||A short description of your handler|
|Author name||Your name|
|Version||Starting version of your handler|
|Has dependencies||Whether your handler has dependencies (|
|Need version bumper||Whether you want to use |
When you see the following message, it means that your handler has been created successfully in that specific handler's path of MindsDB.
+ Sample Handler is created successfully!
If you navigate to the path to your handler, a quick tree view of your handler would be like the following scheme.
<name>_handler/ ├── README.md ├── __about__.py ├── __init__.py ├── <name>_handler.py ├── <name>_tables.py └── icon.svg 1 directory, 6 files
The files that need to be updated are
icon.svg. If your handler has any dependencies, once you've installed the dependencies, don't forget to update the
requirements.txt file and put the output of
pip freeze command into it as follows.
pip freeze > requirements.txt
In the next two sections, we'll be talking about each file and method so that you can write your handlers in a flash!
Develop Your Handler
In this section, we'll be going through both
_tables.py files and their instructions. To run a local instance of MindsDB, create a virtual environment, install the dependencies, and run it with the following commands.
$ virtualenv venv && source venv/bin/activate $ pip install -e . $ pip install -e ".[dev]"
The following command runs the MindsDB engine on your local machine and shows the exact port that it's accessible from.
$ python -m mindsdb
Let's dive into the
<name>_tables.py files that contain most of our implementations. In my case, my handler name is
Sample so I'll bring examples and practices based on this name.
The following handler stub structure is generated in
sample_handler.py by default.
class SampleHandler(APIHandler): def __init__(self, name: str, **kwargs) -> None: ... def check_connection(self) -> StatusResponse: ... def connect(self): ... def native_query(self, query: str) -> StatusResponse: ...
We'll go through each method and the way you should implement them.
SampleHandler.__init__() method gets triggered on the following
CREATE DATABASE query execution over in the dashboard.
CREATE DATABASE sample_db WITH ENGINE = 'sample';
Inside this method, you should create the tables that you need. As you can see, there is already a
SampleTable class created there by default. Create your required tables inside
<name>_tables.py, import them into the
<name>_handler.py and add them in
_tables variables inside the
... from mindsdb.integrations.handlers.sample_handler.sample_tables import SampleTable class SampleHandler(APIHandler): def __init__(self, name: str, **kwargs) -> None: """initializer method Args: name (str): handler name """ super().__init__(name) self.connection = None self.is_connected = False _tables = [ SampleTable, #... ] for Table in _tables: self._register_table(Table.name, Table(self))
Notice that there are two attributes called
self.is_connected inside the initializer method. We need to set them from within the
This method has to return a
StatusResponse(True/False) object meaning the handler is still available to access the third-party for the further communications. This method is also where we set a bool value for
self.is_connected. A quick sample for this method would be like this.
... def check_connection(self) -> StatusResponse: response = StatusResponse(False) try: _ = requests.get("https://google.com", timeout=5).raise_for_status() response.success = True self.is_connected = True except requests.exceptions.RequestException as e: response.error_message = e return response
This method is where you have to put a value into the
self.connection attribute. A good example for this would be returning a
... def connect(self) -> Callable[..., Response]: """making the connectino object Returns: Callable: requests.get """ self.connection = requests.get return self.connection
We need the value that this method puts into the
self.connection later in our
Now, let's see the
sample_tables.py and the methods and classes that live there. You can copy and paste your tables from the initial
SampleTable or even modify it. The first step is to specify each table's columns and names. You can do this by changing the
columns attributes of each class.
class SampleTable(APITable): name: str = "sample" columns: List[str] = ... def __init__(self, handler: APIHandler): ... def select(self, query: ast.Select) -> pd.DataFrame: ... def get_columns(self, ignore: List[str] = ) -> List[str]: ...
There is a
SampleTable.select() method defined by default. This is the only method that you probably need to change. This method is called whenever someone queries a
SELECT statement over your table. All the parameters and clauses that the user specifies can be reachable from the
You can define other methods such as
SampleTable.insert() for the
INSERT INTO purpose.
Don't Forget to..
I highly recommend you take a look over other the handler integrations' source code. They'll give you a nice point of view about the components and how they're supposed to be implemented. As references, I suggest the MediaWiki Handler and PyPI Handler which I maintain!
What's more, Artem Veremey has published "Building a New MindsDB Integration: A Step-by-Step Guide With Examples From the GitHub Handler" article over on his blog that might give you more perspectives and ideas about this integration. Shout out to him!
In this quick tutorial, we talked about MindsDB and the way you can ship your desired handlers over it via a simple Cookiecutter template. Hopefully, this tutorial has helped you build your integrations easier.