Product Improvements in Matillion ETL version 1.40

Here at Matillion, we value user feedback. For each release, we ensure that customer requests are prioritized and introduced into the product. The Product Team at Matillion has been working through our backlog of customer suggested improvements, alongside changes to the third-party services we have connectors for to ensure users are getting an up-to-date and optimized experience on Matillion ETL. We’re happy to announce new product improvements in Matillion ETL version 1.40.

A list of product Improvements in Matillion ETL version 1.40

The below improvements have been released as part of version 1.40 of Matillion ETL which came as requests directly from customers:

“I want to work with Snowflake Streams from Matillion ETL for Snowflake”

One of the reasons customers choose and love Matillion ETL for Snowflake is its purpose-built approach and alignment with Snowflake. Every time Snowflake announces new functionality, customers expect that functionality to be available from Matillion. Snowflake recently announced support for Streams, which allows all inserts/updates and deletes made to a table to be tracked. We have already had requests to Create Streams from Matillion and ingest Stream Data. To meet this request we have added a Create Stream component into Orchestration jobs:

A Stream Input component is now available in Transformation jobs:

“I want to Authenticate to Snowflake with my keypair rather than username and password”

Before version 1.40, Matillion ETL for Snowflake connected to Snowflake using a username and password. However, a customer’s internal security team mandated that keypairs should be used rather than passwords whenever possible. We have built this functionality into Matillion, so now a password type option is available on the Snowflake Connection. The keypair can be stored encoded in the Password Manager, just as the password would be. This functionality provides our customers with additional security requirements, common amongst our large enterprise users, peace of mind.

“Python 3.4 is reaching End of Life soon”

Although Matillion has a host of transformation components and capabilities, there are some use cases where customers require custom Python code. In doing so, customers have many different Python scripts as part of their Matillion jobs and these can be written in Jython, Python 2.6 or Python 3.4. As Python 3.4 has reached End of Life, Matillion v1.40 uses Python 3.6 when Python 3 is selected in the Python component.

“In the JDBC Incremental Loader can we prepopulate the prefix of the URL Field”

Selecting the database type in the Database Query Component prepopulates the prefix of the URL field. In the JDBC Incremental Load Wizard, however, the URL field was previously blank, requiring the user to remember the format of the URL. We have made a change so that the JDBC Incremental Load Wizard now has the same behavior as the Database Query component.

“I can’t see file write progress on the Data Transfer component”

The Data Transfer component replaced the S3 Put and S3 Get components in a previous release of Matillion ETL. For some large files, the component would just run and run and not show progress so it wasn’t clear if the file was copying or if the component was stuck, which caused some confusion for users. We have now added a progress percentage to the Run history:

“Can you improve size of the + and – signs on buttons?”

Customers have feedback to us that at some resolutions the plus ‘+’ and minus ‘-’ symbols on the add and remove buttons weren’t easily visible.

Before	After

Based on customer feedback, we improved the UI by increasing the size of the symbols.

“I want to use the keyboard shortcut Ctrl+A on single-line text input fields”

We understand that some developers want to avoid using the mouse as much as possible. Developers have reached out to say that for single-line text input fields, such as the Name property on most components, Ctrl + A didn’t work. We have added this functionality so you can quickly select all text without touching your mouse.

“I want to manage my variables from within a Python Script component”

In many components where any script is written, it is likely a variable value could be used. In the Calculator component, Matillion shows all variables with the default value and then has a “Manage Variables” button. This button allows users to change or add new variables without having to leave the script editor they are in. We had a request to add this into the Python Script component so we have added this capability where the variables are shown.

“On the Sample tab, I want to edit a filter directly rather than use the pop-up box”

On the Sample Tab of any Transformation component, users can filter the sample data displayed. This is done in a pop-up box. Users have to set the filter, then go back to the Sample Tab, and if another change is needed they have to go back into the pop-up box to edit. We have changed it so the filter logic can be edited in place, providing a more streamlined experience allowing you to adjust filters from the main Sample Tab view.

“I want to use SMB2 or SMB3 protocol in the Data Transfer Component”

The Data Transfer and File Iterator components can now use SMB2 and SMB3 protocols when using Windows Fileshare as the source/destination. The protocol level is automatically negotiated and the highest available one is used, so you don’t need to make any changes in the component. This means Matillion ETL will automatically take advantage of the increased performance and security of SMB2 and SMB3 if the source or destination supports it.

Have an improvement you want to see in the product?

We are always interested in how customers are using Matillion ETL and welcome feedback on how we can improve the product so we can make it better for everyone. Submit your improvement request support@matillion.com.