Contributing Annotations

The simplest type of contributions are the ones that only involve modifications on JSON annotations.

These can modifications can come in different ways:

Creating an annotation for a new primitive

The most usual scenario will be adding a new primitive that does not exist yet in the repository and that can be directly integrated by writing a simple JSON annotation.

In this case, please follow these steps:

1. If it does not exist yet, create a new GitHub issue requesting the new primitive. As indicated previously, provide as many details as possible about the new primitive, like links to the documentation, what it does and what it is useful for.

2. Indicate in the issue description or in a comment that you are available to apply the changes yourself.

3. Wait for the feedback from the maintainers, who will approve the issue and assign it to you, before proceeding to implement any changes. Be open to discuss with them about the need of adding this primitive, as maybe there are other primitive that offer the same functionality, and about the best approach to add it.

4. Once the issue has been approved and assigned to you, implement the necessary changes in your own fork of the project. Please implement them in a branch named after the issue number and title, as this makes keeping track of the history of the project easier in the long run.

   1. Create a new JSON annotation. This can be made from scratch, or you can copy another one and modify it.

   2. The name of the file should correspond to the fully qualified name of the class or function that you are referencing inside the primitive. For example, if you are adding a primitive that uses the class CoolPrimitive from the module super.cool.module, the name of the file should be super.cool.module.CoolPrimitive.json.

   3. Add proper description of what the primitive does in the corresponding entry, as well as a link to its documentation. If there is no documentation available, put the link to its source code. And don’t forget to add you name and e-mail address to the contributors list!

   4. Add a pipeline annotation that uses your primitive inside the pipelines folder, named exactly like your primitive, and test it with the command mlstars test mlstars/pipelines/your.pipeline.json. If adding a pipeline is not possible for any reason, please inform the maintainers, as this probably means that a new dataset needs to be added.

5. Review your changes and make sure that everything continues to work properly by executing the make test-all command.

6. Push all your changes to GitHub and open a Pull Request, indicating in the description which issue you are resolving and what the changes consist of.

Modifying an existing annotation

Sometimes you might think that an existing annotation needs to be modified in some way.

Usually this is because one of the following reasons:

• There is an error in the JSON that prevents it from working properly

• Some hyperparameters are not properly exposed, or not exposed at all

• Documentation is not complete enough

In this case, please follow these steps:

1. Create a new GitHub issue explaining what needs to be changed and why.

2. Indicate in the issue description or in a comment that you are available to apply the changes yourself.

3. Wait for the feedback from the maintainers, who will approve the issue and assign it to you, before proceeding to implement any changes. Be open to discussion, as sometimes you might find out that some of the things that you considered an error are actually intentional. For example, a hyperparameter that you consider missing might have been intentionally left out for performance or compatibility issues.

4. Once the issue has been approved and assigned to you, implement the necessary changes in your own fork of the project. Please implement them in a branch named after the issue number and title, as this makes keeping track of the history of the project easier in the long run. Don’t forget to add you name and e-mail address to the contributors list while you are at it!

5. Make sure that the annotation still works by testing the corresponding pipeline. Normally, this can be done by running the command mlstars test mlstars/pipelines/your.pipeline.json.

6. Review your changes and make sure that everything continues to work properly by executing the make test-all command.

7. Push all your changes to GitHub and open a Pull Request, indicating in the description which issue you are resolving and what the changes consist of.

Creating a new version of an existing annotation

Sometimes you might find that a primitive for a particular annotation already exists, but that modifying it in some way allows adapting it more precisely to some particular scenarios while, at the same time making this unusable for others

Some examples of this would include:

• Use the predict_proba method instead of the predict one in a scikit-learn classifier.

• Alter the hyperparameter ranges to make the primitive more efficient when working with certain type of data or problems.

In this cases, what you need to do is to create a new annotation which is basically a copy of the other one with some modifications.

In this case, please follow these steps:

1. Create a new GitHub issue explaining what needs to be changed and why.

2. Indicate in the issue description or in a comment that you are available to apply the changes yourself.

3. Wait for the feedback from the maintainers, who will approve the issue and assign it to you, before proceeding to implement any changes. As always, be open to discussion, as sometimes you might find that the behavior which you want to cover is already achievable by using certain init_params.

4. Once the issue has been approved and assigned to you, implement the necessary changes in your own fork of the project. Please implement them in a branch named after the issue number and title, as this makes keeping track of the history of the project easier in the long run.

   1. Make a copy of the original JSON annotation.

   2. The name of the file should be the same as the original one, with a suffix added after the last dot . indicating what the changes are. For example, if you are adapting the hyperparameters of the primitive named some.cool.primitive.json to improve its performance on huge datasets, you might name the new file some.cool.primitive.huge_datasets.json.

   3. Apply the necessary changes to the new file and add it to the repository. Don’t forget to add you name and e-mail address to the contributors list while you are at it!

   4. Add a pipeline annotation that uses your primitive inside the pipelines folder, named exactly like your primitive, and test it with the command mlstars test mlstars/pipelines/your.pipeline.json. If adding a pipeline is not possible for any reason, please inform the maintainers, as this probably means that a new dataset needs to be added.

5. Review your changes and make sure that everything continues to work properly by executing the make test-all command.

6. Push all your changes to GitHub and open a Pull Request, indicating in the description which issue you are resolving and what the changes consist of.

Contributing Custom Primitives Contributing Adapters