Lead Data Engineer - Spark/Flink/Scala Engineer
Company: Samsung Research Global
Location: Irvine
Posted on: April 10, 2021
|
|
Job Description:
Irvine , CA Location: Irvine, CA The Visual Display Intelligence
Lab at Samsung Research America is building a next-generation data
platform to support Smart TV products and services. With offices in
Mountain View and Irvine, CA, we are close to a number of our
collaboration partners. Our research and development include TV
analytics, ads targeting, and personalized services. We ingest and
process billions of records daily from millions of TVs in the
field, and we are looking for an experienced professional to join
our team on the development of an integrated data platform. At
Visual Display Intelligence lab, Samsung Research America, we
operate as a start-up while leveraging the broader assets of a
larger global enterprise. We do advance research and development to
immensely benefit our core business verticals in the space of Ads
and Marketing. Our team primarily focuses on processing huge data
sets, build robust and scalable big data pipelines, conduct
advanced research in implementing state-of-the-art ML and AI
techniques, build data services that are both real-time and batch
based. All those combined provide the best-in-class experience to
our customers across consumer electronics and internet of things.
Many of our engineers take pride in publications, filing A-grade
patents and build robust products and services. General Description
We are looking for lead Scala Engineers with experience with batch
and/or streaming jobs. We utilize Spark for batch jobs and Flink
for real-time streaming jobs. Experience with Hadoop, Hive, AWS S3
is also an asset. Responsibilities Create new, and maintain
existing Spark/Flink jobs written is Scala Produce unit and system
tests for all code Lead architecture / design discussions to
improve our existing frameworks Define scalable calculation logic
for interactive and batch use cases Interact with infrastructure
and data teams to produce complex analysis across data Interact
with product managers to understand requirements, come up with
estimates and take it to delivery. The ability to lead senior
engineers. This role has the growth potential of managing a few
senior engineers and engineers in distributed locations. Required
Qualifications: Bachelors degree in Computer Science / MS or
equivalent combination of education, training, and experience. 8+
years of experience with Scala and/or Java. Required experience
with Hadoop, Spark 2 years of experience in people management is
preferred Knowledge and experience with cloud-based technologies
Experience in batch or real-time data streaming Ability to
dynamically adapt to conventional big-data frameworks and open
source tools if project demands Knowledge of design strategies for
developing scalable, resilient, always-on data lake Must be very
comfortable with reading and writing Scala code An aptitude for
analytical problem solving Deep knowledge of troubleshooting and
tuning Spark applications and Hive scripts to achieve optimal
performance Good understanding/knowledge of HDFS architecture and
various components such as Job Tracker, Task Tracker, Name Node,
Data Node, HDFS high availability (HA) and Map Reduce programming
paradigm. Experienced working with various Hadoop Distributions
(Cloudera, Hortonworks, MapR, Amazon EMR) to fully implement and
leverage new Hadoop features Experience in developing Spark
Applications using Spark RDD, Spark-SQL, Spark -Yarn, Spark Mlib
and Data frame APIs Experience with real-time data processing and
streaming techniques using Spark streaming and Kafka, moving data
in and out HDFS and RDBMS. Experience with ML/distributed ML
frameworks like Spark-MLlib, Tensorflow etc. Model training with
batch and real-time prediction scenarios : Use machine language and
statistical modeling techniques such as decision trees, logistic
regression, Bayesian analysis and others to develop and evaluate
algorithms to improve product/system performance, quality, data
management and accuracy. Familiarity with open source configuration
management and development tools Ability to work with cross
functional teams from requirements to delivery and drive projects.
Preferred Qualifications: Hands on experience and production use of
Hadoop/Cassandra, Spark, Flink and other distributed technologies
would be a plus Other Technologies Scalatest Gradle/Maven Airflow
SQL Redshift Samsung is an EEO/Veterans/Disabled/LGBT employer. We
welcome and encourage diversity as we strive to create an inclusive
workplace. Additional Information for California Residents: *
Information about what we are doing to meet the obligations of the
California Consumer Privacy Act (CCPA) and other important
information for California residents is available at SRAs Job
Applicant Privacy Policy . Please review the policy carefully, and
if you agree to the collection, processing, use, and transfer of
your personal information as described in the policy, please
indicate your acceptance by selecting Accept from the following
drop down menu. Please select U.S. Equal Opportunity Employment
Information (Completion is voluntary) Individuals seeking
employment at Samsung Research America are considered without
regards to race, color, religion, national origin, age, sex,
marital status, ancestry, physical or mental disability, veteran
status, gender identity, or sexual orientation. You are being given
the opportunity to provide the following information in order to
help us comply with federal and state Equal Employment
Opportunity/Affirmative Action record keeping, reporting, and other
legal requirements. Completion of the form is entirely voluntary.
Whatever your decision, it will not be considered in the hiring
process or thereafter. Any information that you do provide will be
recorded and maintained in a confidential file. Gender Please
select Are you Hispanic/Latino? Please select Race & Ethnicity
Definitions If you believe you belong to any of the categories of
protected veterans listed below, please indicate by making the
appropriate selection. As a government contractor subject to
Vietnam Era Veterans Readjustment Assistance Act (VEVRAA), we
request this information in order to measure the effectiveness of
the outreach and positive recruitment efforts we undertake pursuant
to VEVRAA. Classification of protected categories is as follows: A
"disabled veteran" is one of the following: a veteran of the U.S.
military, ground, naval or air service who is entitled to
compensation (or who but for the receipt of military retired pay
would be entitled to compensation) under laws administered by the
Secretary of Veterans Affairs; or a person who was discharged or
released from active duty because of a service-connected
disability. A "recently separated veteran" means any veteran during
the three-year period beginning on the date of such veteran's
discharge or release from active duty in the U.S. military, ground,
naval, or air service. An "active duty wartime or campaign badge
veteran" means a veteran who served on active duty in the U.S.
military, ground, naval or air service during a war, or in a
campaign or expedition for which a campaign badge has been
authorized under the laws administered by the Department of
Defense. An "Armed forces service medal veteran" means a veteran
who, while serving on active duty in the U.S. military, ground,
naval or air service, participated in a United States military
operation for which an Armed Forces service medal was awarded
pursuant to Executive Order 12985. Veteran Status Please select
Form CC-305 OMB Control Number 1250-0005 Expires 05/31/2023
Voluntary Self-Identification of Disability Why are you being asked
to complete this form? We are a federal contractor or subcontractor
required by law to provide equal employment opportunity to
qualified people with disabilities. We are also required to measure
our progress toward having at least 7% of our workforce be
individuals with disabilities. To do this, we must ask applicants
and employees if they have a disability or have ever had a
disability. Because a person may become disabled at any time, we
ask all of our employees to update their information at least every
five years. Identifying yourself as an individual with a disability
is voluntary, and we hope that you will choose to do so. Your
answer will be maintained confidentially and not be seen by
selecting officials or anyone else involved in making personnel
decisions. Completing the form will not negatively impact you in
any way, regardless of whether you have self-identified in the
past. For more information about this form or the equal employment
obligations of federal contractors under Section 503 of the
Rehabilitation Act, visit the U.S. Department of Labors Office of
Federal Contract Compliance Programs (OFCCP) website at
www.dol.gov/ofccp . How do you know if you have a disability? You
are considered to have a disability if you have a physical or
mental impairment or medical condition that substantially limits a
major life activity, or if you have a history or record of such an
impairment or medical condition. Disabilities include, but are not
limited to: Autism Autoimmune disorder, for example, lupus,
fibromyalgia, rheumatoid arthritis, or HIV/AIDS Blind or low vision
Cancer Cardiovascular or heart disease Celiac disease Cerebral
palsy Deaf or hard of hearing Depression or anxiety Diabetes
Epilepsy Gastrointestinal disorders, for example, Crohn's Disease,
or irritable bowel syndrome Intellectual disability Missing limbs
or partially missing limbs Nervous system condition for example,
migraine headaches, Parkinsons disease, or Multiple sclerosis (MS)
Psychiatric condition, for example, bipolar disorder,
schizophrenia, PTSD, or major depression Disability Status Please
select 1 Section 503 of the Rehabilitation Act of 1973, as amended.
For more information about this form or the equal employment
obligations of Federal contractors, visit the U.S. Department of
Labor's Office of Federal Contract Compliance Programs (OFCCP)
website at www.dol.gov/ofccp . PUBLIC BURDEN STATEMENT: According
to the Paperwork Reduction Act of 1995 no persons are required to
respond to a collection of information unless such collection
displays a valid OMB control number. This survey should take about
5 minutes to complete.
Keywords: Samsung Research Global, Irvine , Lead Data Engineer - Spark/Flink/Scala Engineer, Engineering , Irvine, California
Click
here to apply!
|
Didn't find what you're looking for? Search again!
Other Engineering JobsQA Engineer (automation) Description: Dave is a banking app on a mission to create financial opportunity that advances America's collective potential. Dave's financial tools, including its debit card and spending account, help its more than (more...) Company: Dave Location: Los Angeles Posted on: 04/18/2021 Senior Electronics Products Array Mechanical Engineering Manager Description: Position Overview:Boeing Defense Space Security is seeking an Electronics Products Array Mechanical Engineering Manager Level L - to manage-engineering teams-designing future generations of satellites (more...) Company: BOEING Location: Sylmar Posted on: 04/18/2021 Field Applications Engineer (Embedded) Description: Position:Field Applications Engineer Embedded Job Description:What You'll Be Doing ul li Responsible for utilizing an advanced understanding of the customer needs and technical leadership expertise (more...) Company: Arrow Electronics Location: Woodland Hills Posted on: 04/18/2021 Experienced Satellite DSP Subsystems and Algorithms Engineer Description: As an Experienced DSP Satellite Subsystems and Algorithms Engineer Level 4 , you will design and develop state-of-the-art, space-based communication systems for commercial, civil and/or national security (more...) Company: BOEING Location: Chatsworth Posted on: 04/18/2021 Sr Data Warehouse Engineer Description: Assists with analyzing complex business problems to be solved with automated systems. As part of a software project implementation team, provides technical expertise in identifying, evaluating and developing (more...) Company: Oracle Location: El Segundo Posted on: 04/18/2021 DevOps Engineer Description: Position Overview Cast and Crew is looking for an experienced DevOps Engineer to play a key leadership role in our team as we automate our multiple development pipelines that flow into AWS, Azure, and (more...) Company: Cast & Crew Entertainment Services, LLC Location: Los Angeles Posted on: 04/18/2021 Critical Infrastructure Engineer - Los Angeles, CA or Sunnyvale, CA Description: About Lumen br Lumen is guided by our belief that humanity is at its best when technology advances the way we live and work. With 450,000 route fiber miles serving customers in more than 60 countries, (more...) Company: Lumen Location: Los Angeles Posted on: 04/18/2021 Controls Engineer (Design Group) Description: About Us:Design Group operates from more than 45 offices in the United States and India, providing engineering, consulting and technical services for the world's leading companies in the food and beverage, (more...) Company: Barry Wehmiller Design Group Location: Los Angeles Posted on: 04/18/2021 IOS Engineer Description: A company located in West LA founded with a vision to create exceptional products that inspire people to indulge while doing good in the world is expanding their engineering team. They are committed to (more...) Company: Motion Recruitment Location: El Segundo Posted on: 04/18/2021 Senior Data Engineer at Major Video Streaming Service Description: Contract - 65-75/hr - Los Angeles,CaliforniaJob Description Major video streaming service is looking to bring on more Data Engineers to their team. You will be working on building new Data Pipelines (more...) Company: Motion Recruitment Partners Location: Los Angeles Posted on: 04/18/2021 |