Please Sign In or Register to join the Community to see more content!
New version of Unify - 16 Feb 2023
We've updated Unify with a new version that includes improved support for large datasets and PI Asset Framework (AF) as well as other enhancements. There’s nothing specific you need to do to take advantage of these features - they’re already enabled and working in the background. Note - the new version will be launched at 7pm Eastern today.
The new version includes:
Smart retry and load management for long-running Pipelines:
- Better processing of large data uploads
- Separate uploads into “chunks” to streamline data processing
- Improved memory use when processing large data uploads
- Updated cache keys for a 80% reduction in memory usage for a Pipeline
- Improved reliability under load
- Introduced a new event type for handling multiple appends without reprocessing the whole dataset for each append
- Handle error conditions to ensure Pipelines continue running
- Added a centralized queue of evergreen Pipeline jobs, minimizing issues between multiple Pipelines. The central queue system will eventually cancel jobs that fail a certain number of times (5 is the current value for MAX_RETRIES)
Enhancements to PI Asset Framework (AF) data model support:
- Support for attributes with Analysis Data References
- Support for special characters with the encoding set to UTF-8
Bug Fixes:
- Corrected dataset loading / caching issue when running over 60 seconds
- Updated CloudFormation template to remove a bad default for the Unify URL
Let us know if you have questions or comments!
Tagged:
