hub / github.com/WeBankFinTech/Exchangis
github.com/WeBankFinTech/Exchangis @release-1.1.8
Chat with this repo
7,112 symbols
19,574 edges
865 files
1,572 documented · 22%
What it actually does AI analysis from the code graph — generated when you open this
loading…
README
Exchangis
English | 中文
介绍
Exchangis是微众银行大数据平台 WeDataSphere 与社区用户共同研发的的新版数据交换工具，支持异构数据源之间的结构化和非结构化数据传输同步。
Exchangis 抽象了一套统一的数据源和同步作业定义插件，允许用户快速接入新的数据源，并只需在数据库中简单配置即可在页面中使用。
基于插件化的框架设计，及计算中间件 Linkis，Exchangis 可快速集成对接 Linkis 已集成的数据同步引擎，将 Exchangis 的同步作业转换成 Linkis 数据同步引擎的数据同步作业。
借助于 Linkis 计算中间件的连接、复用与简化能力，Exchangis 天生便具备了高并发、高可用、多租户隔离和资源管控的金融级数据同步能力。
界面预览
核心特点
1. 轻量化的数据源管理
-
基于 Linkis DataSource，抽象了底层数据源在 Exchangis 作为一个同步作业的 Source 和 Sink 所必须的所有能力。只需简单配置即可完成一个数据源的创建。
-
特别数据源版本发布管理功能，支持历史版本数据源回滚，一键发布无需再次配置历史数据源。
2. 高稳定，快响应的数据同步任务执行
-
近实时任务管控
快速抓取传输任务日志以及传输速率等信息，对多任务包括CPU使用、内存使用、数据同步记录等各项指标进行监控展示，支持实时关闭任务； -
任务高并发传输
多任务并发执行，并且支持复制子任务，实时展示每个任务的状态，多租户执行功能有效避免执行过程中任务彼此影响进行； -
任务状态自检
监控长时间运行的任务和状态异常任务，中止任务并及时释放占用的资源。
3. 与DSS工作流打通，一站式大数据开发的门户
- 实现DSS AppConn包括一级 SSO 规范，二级组织结构规范，三级开发流程规范在内的三级规范；
- 作为DSS工作流的数据交换节点，是整个工作流链路中的门户流程，为后续的工作流节点运行提供稳固的数据基础；
4. 支持多种导数引擎
- 支持Sqoop和DataX引擎进行多种异构数据源之间的导数
整体设计
架构设计
相关文档
交流贡献
如果您想得到最快的响应，请给我们提 issue，或者扫码进群：
License
Exchangis is under the Apache 2.0 License. See the License file for details.
Extension points exported contracts — how you extend this code
RegisterMetricConverterFactory (Interface)Registrable Converter factory @param [7 implementers]
exchangis-job/exchangis-job-server/src/main/java/com/webank/wedatasphere/exchangis/job/server/metrics/converter/RegisterMetricConverterFactory.java
ExchangisEvent (Interface)
Event between different modules [6 implementers]
exchangis-job/exchangis-job-common/src/main/java/com/webank/wedatasphere/exchangis/job/listener/ExchangisEvent.java
ProjectCopyService (Interface)
@author tikazhang @Date 2022/4/24 21:11 [7 implementers]
exchangis-project/exchangis-project-provider/src/main/java/com/webank/wedatasphere/exchangis/project/provider/service/ProjectCopyService.java
EngineSettingsDao (Interface)
Engine settings dao [8 implementers]
exchangis-engines/exchangis-engine-core/src/main/java/com/webank/wedatasphere/exchangis/engine/dao/EngineSettingsDao.java
JobPluginCollector (Interface)
Created by jingxing on 14-9-9. [9 implementers]
exchangis-engines/engines/datax/datax-core/src/main/java/com/alibaba/datax/common/plugin/JobPluginCollector.java
ExchangisDataSource (Interface)
Data source common basic inf TIPS: Not need to define modelId [8 implementers]
exchangis-common/src/main/java/com/webank/wedatasphere/exchangis/common/domain/ExchangisDataSource.java
Counter (Interface)
An incrementing and decrementing counter metric. 计数器型指标，适用于记录调用总量等类型的数据 [5 implementers]
exchangis-job/exchangis-job-metrics/src/main/java/com/webank/wedatasphere/exchangis/metrics/api/Counter.java
DataSourceService (Interface)
Data source service api TODO define all inf in ExchangisDataSourceService [3 implementers]
exchangis-datasource/exchangis-datasource-service/src/main/java/com/webank/wedatasphere/exchangis/datasource/service/DataSourceService.java
Core symbols most depended-on inside this repo
getcalled by 582
exchangis-engines/engines/datax/datax-core/src/main/java/com/alibaba/datax/plugin/unstructuredstorage/reader/UnstructuredStorageReaderUtil.java
format
called by 550
exchangis-engines/engines/datax/datax-core/src/main/java/com/alibaba/datax/plugin/rdbms/util/SqlFormatUtil.java
asDataXException
called by 543
exchangis-engines/engines/datax/datax-core/src/main/java/com/alibaba/datax/common/exception/DataXException.java
error
called by 462
exchangis-job/exchangis-job-server/src/main/java/com/webank/wedatasphere/exchangis/job/server/log/JobServerLogging.java
add
called by 461
exchangis-datasource/exchangis-datasource-service/src/main/java/com/webank/wedatasphere/exchangis/datasource/service/RateLimitService.java
info
called by 355
exchangis-job/exchangis-job-server/src/main/java/com/webank/wedatasphere/exchangis/job/server/log/JobServerLogging.java
getString
called by 351
exchangis-engines/engines/datax/datax-core/src/main/java/com/alibaba/datax/common/util/Configuration.java
getValue
called by 313
exchangis-job/exchangis-job-common/src/main/java/com/webank/wedatasphere/exchangis/job/domain/params/JobParam.java
Shape
Languages
Java98%
TypeScript1%
Python1%
Modules by API surface
exchangis-engines/engines/datax/datax-core/src/main/java/com/alibaba/datax/common/statistics/PerfTrace.java117 symbols
web/src/common/service.js64 symbols
exchangis-job/exchangis-job-launcher/src/main/java/com/webank/wedatasphere/exchangis/job/launcher/domain/LaunchableExchangisTask.java60 symbols
exchangis-engines/engines/datax/datax-core/src/main/java/com/alibaba/datax/common/util/Configuration.java53 symbols
exchangis-common/src/main/java/com/webank/wedatasphere/exchangis/dao/domain/ExchangisJobParamConfig.java51 symbols
exchangis-job/exchangis-job-common/src/main/java/com/webank/wedatasphere/exchangis/job/domain/SubExchangisJob.java43 symbols
exchangis-datasource/exchangis-datasource-core/src/main/java/com/webank/wedatasphere/exchangis/datasource/core/domain/ExchangisDataSourceItem.java43 symbols
exchangis-job/exchangis-job-common/src/main/java/com/webank/wedatasphere/exchangis/job/vo/ExchangisJobVo.java42 symbols
exchangis-privilege/exchangis-privilege-common/src/main/java/com/webank/wedatasphere/exchangis/privilege/domain/PermissionReportData.java41 symbols
exchangis-engines/engines/datax/datax-core/src/main/java/com/alibaba/datax/core/job/JobContainer.java38 symbols
exchangis-job/exchangis-job-server/src/main/java/com/webank/wedatasphere/exchangis/job/server/builder/engine/DataxMappingContext.java37 symbols
exchangis-datasource/exchangis-datasource-core/src/main/java/com/webank/wedatasphere/exchangis/datasource/core/vo/DataSourceCreateVo.java37 symbols
Datastores touched
(mysql)Database · 1 repos
exchangis_v3Database · 1 repos
For agents
$ claude mcp add Exchangis \
-- python -m otcore.mcp_server <graph>