Interconnected Among All Products
Unlike traditional data integration tools, which tend to have a
fragmented organizational structure, EsDataFactory adopts a logical, layered management
model for data warehousing. It includes built-in layers such as ODS, Integration Layer,
Summary Layer, and Data Marts. Users can freely customize their configuration based on
actual needs. Additionally, unique functional guides are designed to align with the
characteristics of each data warehouse layer, enabling users to quickly become proficient
in the tool.
Multi-Dimensional and Multi-Method Visualization Modeling
EsDataFactory provides a visual modeling interface for creating data
models, supporting both custom model development and field selection from existing data
structures (such as database tables, views, files, etc.). It offers a global view to
visualize the relationships between models and between models and dimensions. It allows
users to build Normal Form schema, star schema, and snowflake schema.
Personalized Model Resource Retrieval
EsDataFactory features a search engine-like retrieval system, offering
both simple and advanced search modes. The simple mode allows users to search for models
based on the English or Chinese names of the model and its fields, with matching items
highlighted. The advanced mode enables more complex searches, tailored to user-defined
parameters, facilitating efficient retrieval of model resources.
Incremental Capture and Data Loading
EsDataFactory offers various data capture mechanisms for change data
capture, such as timestamp-based, M05, triggers, and full table comparisons. These
processes are transparent to the user, requiring only simple configurations to capture
incremental data. The platform supports multiple data offloading strategies, including
data overwrite, data append, data update, and update-insert, ensuring comprehensive
coverage of all data offloading scenarios.
Drag-and-Drop Process Designer
EsDataFactory features a fully drag-and-drop process designer, enabling
users to complete complex ETL tasks and job workflow definitions by simply dragging and
dropping data resources and transformation components. With zero coding required, the
interface is user-friendly, easy to read, and easy to maintain. The comprehensive
graphical editing functions, such as copy, paste, undo, redo, and auto-align, enhance
usability and efficiency.
Visual Debugging and Pre-Loading
The platform supports visual debugging, including step-by-step execution,
execute to functionality. Users could check the status of each step, execution time, and
result sets of execution. For developers, EsDataFactory offers pre-loading mechanisms and
the execution status of each step, making it easier to quickly validate workflows and
debug scripts, streamlining the development process.
Rich Data Processing Components
EsDataFactory offers various data capture mechanisms for change data
capture, such as timestamp-based, M05, triggers, and full table comparisons. These
processes are transparent to the user, requiring only simple configurations to capture
incremental data. The platform supports multiple data offloading strategies, including
data overwrite, data append, data update, and update-insert, ensuring comprehensive
coverage of all data offloading scenarios.
Drag-and-Drop Process Designer
EsDataFactory features a fully drag-and-drop process designer, enabling
users to complete complex ETL tasks and job workflow definitions by simply dragging and
dropping data resources and transformation components. With zero coding required, the
interface is user-friendly, easy to read, and easy to maintain. The comprehensive
graphical editing functions, such as copy, paste, undo, redo, and auto-align, enhance
usability and efficiency.
Visual Debugging and Pre-Loading
The platform supports visual debugging, including step-by-step execution,
execute to functionality. Users could check the status of each step, execution time, and
result sets of execution. For developers, EsDataFactory offers pre-loading mechanisms and
the execution status of each step, making it easier to quickly validate workflows and
debug scripts, streamlining the development process.
Rich Data Processing Components
EsDataFactory offers a wide range of data processing components to handle
tasks such as data transfer, cleansing, transformation, and loading and offloading. The
platform features a flat icon design, along with comprehensive online help manuals and a
case library, making it easy for users to get started.
Specialized Data Warehouse Application Components
In addition to traditional data processing components, EsDataFactory
encapsulates commonly used data warehouse transformation operations, such as
row-to-column, column-to-row, row record splitting, consistent dimension transformations,
and slowly changing dimension transformations.
“Zero” Expression Cleaning Component
EsDataFactory includes a variety of cleaning rules to perform field-level
and record-level data cleaning and transformation. This eliminates the need for complex
expression definitions. Additionally, the cleaning component supports real-time previews
of data changes before and after cleaning while configuring, and supports automatic
merging of multiple rules for execution, thus saving performance overhead.
Practical Batch Processing
For commonly used data warehouse operations, EsDataFactory offers a wide
range of batch definitions and shortcut operations. The wizard-style guidance helps users
quickly define data models and load data in bulk.
Isolation of Design and Runtime Perspectives
To ensure the stability and reliability of the runtime environment,
EsDataFactory provides a mechanism that isolates the design area from the runtime area.
All job revisions and changes are made in the design area and, after debugging and test
runs, are released to the stable runtime environment through the release mechanism,
ensuring that development and execution proceed without interference.
Scientific Data Security and Anonymization Methods
Protecting sensitive private data is crucial for information security. A
breach of confidential data can lead to severe consequences. EsDataFactory provides a
common data masking algorithms package to ensure data association rules are preserved,
while ensuring equivalence between development and production environments in terms of
data usage.
Common anonymization algorithms include:
Fixed value replacement, prefix and suffix addition, data obfuscation, data trimming,
segment replacement, MD5 processing, hashing, random filling, and offset encryption.
Flexible Scheduling Service
EsDataFactory supports scheduling mechanisms based on time or events,
such as: any time interval, file arrival, script events, etc. The scheduling settings
support crontab expressions, allowing for fine-grained control of schedule timing,
including by month, day, hour, or minute. It also supports scheduling based on weeks. and
also allows setting a time window for scheduling, with adjustable retry intervals.
Powerful Resource Backup and Recovery
EsDataFactory offers a multi-level resource-based backup mechanism,
including the backup of resources such as data models, ETL processes, ETL process flows,
and scheduling services. Backups can be manually triggered or automatically completed on a
scheduled basis by the system. After the backup, the data package can be selectively
restored, which is particularly important for system upgrades, migration, and rapid
failure recovery.
Multi-Angle Operation Monitoring
EsDataFactory supports both overview of execution monitoring and detailed
process monitoring, allowing different levels of operations personnel to manage the system
effectively. It provides various graphical and list-based methods to monitor operational
status, execution duration, process execution, and more. It is capable of tracking the
execution of each ETL process and even each individual transformation.
Comprehensive Resource Monitoring System
EsDataFactory provides a comprehensive resource monitoring system that
tracks system server resource usage, system access status, operation time consumption,
system occupancy, and storage usage. By monitoring a series of resources, potential system
issues can be detected and resolved promptly, ensuring the smooth operation of the system.
Multi-Level Login Authentication
EsDataFactory employs dual authentication mechanisms, including Ukey
signature and user role permissions. This approach controls access based on the source of
the request, preventing unauthorized access and reducing the risk of security incidents.
400-0011-866
400-0011-866
