Derived Datasets are extremely useful outputs in Unify. Their power comes from the flexibility to be consumed by other Asset Pipelines, ODBC/JDBC connections, and connectors in the Connector Portal. A common mistake though, is changing the schema associated with the data flow going into a Derived Dataset. The "schema"…
Unify provides various built-in transformations to transform data into a more usable/context-rich format. Among those transformations is the SQL query where you can perform more advanced/flexible operation on the data. Today, I'm sharing a cheat sheet of the most commonly used SQL queries that our team and customers have…
A recent Element Unify update addresses an issue with how special columns are handled by SQL Query, SQL Join, and Union transformations. This fix prevents unexpected behavior when building graphs but may affect some pipelines that were built before it was deployed. If you see an error like "<Column Name> has conflicting…
It would be pretty neat if there was a way to store favorite or custom transforms. This would save on having to reference previous asset pipelines and rebuild them again. These custom (or myFavorite style) transforms can be dragged and dropped from the snipped queue into the field of view for immediate use.
Replace functions are commonly used to find specific values and replace them with other values. A simple example is to find a state abbreviation (TX) and replace it with it's full name (Texas). In data practices, replace functions are used to help clean or standardize data (i.e., remove unwanted characters, standardize…
The Validator transformation can be a bit non-intuitive to use - even I get puzzled sometimes. But there's a small trick I use to make it much easier to understand. Before I place the Validator transformation in my pipeline I add a SQL Query transformation - which I use to get the number of rows in the data flow. The SQL…
Recursive templates allow you to build component-based models in Unify. I often find it hard to explain what they are / how to use them without bringing up a real-world example. My go-to example is a Generator, since it's comprised of multiple components that most people are familiar with. Below is a conceptual drawing of…
Ever wanted to find a quick way to increase your match rate when unifying your data? I recently came across a cool way to use text analysis to do just that. Check it out! I started with a query on my tag list (using SQL transformation), to count the frequency of all the different words in my tag description field: SELECT…
