G Talent Top
Jobs
Marketing Division/Data Strategy Department/DWH Engineer/✦Famous Digital Media Company✦

Marketing Division/Data Strategy Department/DWH Engineer/✦Famous Digital Media Company✦

Tokyo

Fulltime

Remote

Large Scale Company

Own Products/Services

Global Business

Industry: Internet Conglomerate Company

IT Skills: Java, Azure, Typescript, GCP(Google Cloud Platform), AWS, SQL, Javascript, Python

Working hours: Flextime System

Salary: 550 Million yen〜800 Million yen

Job Description: [Recruitment Background] The company's Data Division promotes the use of data to achieve steady and repeatable business growth through scientific business analysis, forecasting, and improvement. As the company's phrase "we do everything," there are several businesses that can be expected to contribute to business through the use of data, and the company now boasts approximately 40 services, 30 million members, and 2.5 billion monthly PVs, making it a huge data-holding company. The number of users of the company's data analysis environment is more than 1,400. The DWH team is a small, elite group of two people who carry out the company's data lake and DWH management, which supports the foundation of the data analysis and utilization experience in such environment. They are looking for people who can help them improve the scalability and efficiency of both systems and operations by leveraging data engineering + cloud + CI/CD to deliver a better analysis and data utilization experience to their 1,400+ internal analysis environment users. [Overview] About the company Since its establishment in 1998, the company has been investing in businesses that it sees as the future, regardless of size or genre, with sales of 234 billion yen by 2020. In the 22 years since its founding, they have grown steadily and currently operates 58 businesses through more than 20 group companies. The company's group continues to enter and challenge every possible field as a business platform The company is a part of the Group's business, which includes video, e-books, mail-order and other EC services, as well as fintech, solar and English conversation services. [About the Marketing Division] The Marketing Division is an organization of specialists consisting of the Data Infrastructure, Data Science, and Marketing Technology departments, who work to achieve steady and reproducible business growth through scientific business analysis, forecasting, and improvement, which is part of the measures to realize the tech vision being promoted by the company's development organization. [About the DWH Team] The DWH team manages the data lake and DWH used by the entire Group, which is the foundation of data utilization. In order to provide the best analysis and data utilization experience to more than 1,400 users of the internal analysis environment, a small team of two people is in charge of activities to deliver more than 100 DBs and 1,000 tables to the environment where they can be analyzed. In order to support the core of data utilization with a small team of elites, they are promoting system development and operation that fully utilizes the modern cloud environment, and the introduction of a system that automatically detects deterioration in data quality (anomaly detection by expanding metadata). They are looking for strong people who can work with them to deliver better data analysis and data utilization experiences. [Job Description] Establishment of data infrastructure with a focus on GCP * Data flow (ETL, workflow, data validation, data quality check) * Data governance (metadata management, data security management) [Attraction of this position?] ■Mission Support the Group's data strategy through the provision of DWH infrastructure * Save labor in importing data from each business and make the analysis platform available without lead time. * Collect and analyze technical metadata of DWH to detect and prevent data anomalies * Collect and provide business metadata for DWH to improve analyst productivity * Collect and provide DWH business metadata to improve analyst productivity * Optimize costs in a way that does not compromise DX for DWH users [Attraction] * You will be able to face the unique challenges of a large scale DWH infrastructure consisting of many businesses. * This is a great opportunity to work on a large scale DWH platform with many businesses. * This is a great opportunity to work with a large team of people who have a lot of experience in the field of data infrastructure operations. (The contact team is also within the same organization, so they are not in a situation where they cannot pick up the needs of the users. Development environment/team structure 【Team structure】 1 TL, 2 engineer (The contact team is also within the same organization, so there is no situation where they cannot pick up the needs of users. In addition to mid-career hires, new graduates will also be assigned to the team. 【Development environment/tools】 * Mac (JIS or US layout), Windows: You can choose your favorite machine. * Dual displays provided * The standard machine specifications are CPU (Core i5) and memory (16GB), but higher specifications can be selected depending on the development environment required. 【Example of main technology】 [Languages] Python, Hive, Spark [AWS] EMR, Glue, Athena, Lambda, APIGateway [CI, CD, etc.] Terraform, CircleCI, GithubActions, digdag, embulk

Required Skills: [Required] * Experience building products for business use in the cloud, such as AWS and GCP. * Experience designing, developing, and operating data pipelines [Preferred] * Experience in designing and building a product from scratch * Experience in implementing specific measures for any of the knowledge areas defined in the DMBOK [Ideal Applicants] * Experience with dataset and table design in BigQuery, including selection of normalization, denormalization, etc. * Knowledge of data management/DataQuality/DataProfiling * Experience in implementing specific measures for any of the knowledge areas defined in DMBOK * Deep knowledge of Analytics products and experience using them in AWS, GCP, etc.

Required Language Skills