Flink SQL 的 Lookup Join 通过 FOR SYSTEM_TIME AS OF 语法实现流表与维表的关联,用于在实时计算中补充元数据信息。以 OpenAPI 网关监控为例:
维表定义(MySQL 存储网关集群信息):
CREATE TABLE gateway_cluster_dim (
local_ip STRING,
app_group STRING,
region_id STRING,
PRIMARY KEY (local_ip) NOT ENFORCED
) WITH (
'connector' = 'jdbc',
'lookup.cache.max-rows' = '50000',
'lookup.cache.ttl' = '10min',
'lookup.max-retries' = '3'
);
关联查询:
FROM openapi_log_source l
LEFT JOIN gateway_cluster_dim FOR SYSTEM_TIME AS OF l.ts AS g
ON l.LocalIp = g.local_ip
缓存策略选择非常关键:对于变化频率低的小表(如集群信息)采用 ALL 策略,启动时全量加载并定时刷新;对于数据量大的表(如租户信息)采用 LRU 策略,缓存热点数据并设置合适的 TTL(如 10 分钟)以平衡命中率和数据新鲜度。