Ep2. Data Engineering at Uber and Lyft

这一期节目我们和嘉宾泉来聊一聊Data Engineering,在Uber和Lyft的应用和工作体验,对于行业发展的反思与展望,以及对于入行Data Engineering的建议。只要你对Large Scale Data Engineering感兴趣,本期节目都会对你有帮助 :) Timestamps: * 00:00:00 Intro * 00:00:40 Data Infrastructure vs Data Engineer * 00:02:45 Data Engineering 领域介绍 * 00:09:00 Pipeline权限设置 * 00:11:00 Pipeline在Uber的使用场景 * 00:11:45 如何追踪Data Owner * 00:11:52 如何保护用户数据隐私 * 00:17:16 Data Infra转到Data Engineer的日常工作 * 00:21:44 Data Quality Tier * 00:32:52 Uber vs Lyft 数据量级和迭代的区别 * 00:35:46 COVID影响 * 00:38:40 对颠覆性产业和商业模式的反思 * 00:40:27 规模效应和数据 * 00:41:26 ML Infra vs Data Infra * 00:44:47 对新入行Data Engineering的建议 * 00:47:28 联系方式 Links: * Airflow: https://airflow.apache.org/ * Piper: https://eng.uber.com/no-code-workflow-orchestrator/ * uWorc: https://eng.uber.com/no-code-workflow-orchestrator/ * Amundsen: https://eng.lyft.com/open-sourcing-amundsen-a-data-discovery-and-metadata-platform-2282bb436234 * PII: https://en.wikipedia.org/wiki/Personal_data * GDPR: https://en.wikipedia.org/wiki/General_Data_Protection_Regulation * ETL: https://en.wikipedia.org/wiki/Extract,_transform,_load * 联系泉来:[email protected] 联系方式: * 官网: eng.cafe/ * 微信公众号: Eng Cafe * Twitter: @engcafefm * Youtube: Eng Cafe * Email: [email protected] 收听渠道: * Apple Podcast * Google Podcasts * Overcast * Spotify * Pocket Casts * Breaker * Castro * Castbox * Podcast Addict * Player FM * RSS Feed

49分钟
99+
4年前
EarsOnMe

加入我们的 Discord

与播客爱好者一起交流

立即加入

播放列表

自动播放下一个

播放列表还是空的

去找些喜欢的节目添加进来吧