A/B Testing

A/B testing is a statistical framework that helps validate an idea or a theory through data.

For example, a product manager wants to know if changing the color of a buy button from green to blue can encourage more purchases. As a data scientist, it is your job to work with the product manager and, quite often, the engineering team(can help implement the testing settings) to develop a testing plan.

You need to decide at least how many people will see the different colors of the button (sample size), and how many days will the testing run (usually multiples of a week, 7 days), and where should it be running (the US only, or some other small countries just in case testing group is a failure, you don’t want to have a very negative impact to the revenue).

The key assumption of A/B testing is that the control group and the testing group have to be independent. You will probably be asked several questions around this assumption.

You will also need to understand key concepts such as novelty effectlearning effectA/A testing, Simpson’s paradox, etc.

Sample question

The engineering team just invented a people-you-may-know widget. If it is implemented, a user will see their friends on the right-left corner of their homepage. How do you design an experiment to decide whether we should launch this feature or not?


  • Founder of instamentor.com, posturenet.app and sqlpad.io
  • Most recently a senior manager at Apple, where I built and hired a world-class machine learning team of data scientists, software engineers, data engineers, and product managers.
  • Head of data science at Chegg, research scientist at Amazon, serial entrepreneur.
$ 399 ⏰ 1 hour

Get access to Leon Wei's calendar after Stripe process your payment, fully refundable if not 100% satisfied.

⏰ A/B Testing with Leon Wei for $399

  • I am an analytics veteran with over twenty years of experience in driving business value from data.
  • Prior to launching my own practice Intellego Consulting, I built and lead many high-performing data analytics teams at marquee global technology companies. As a player-coach, I have hands-on knowledge of how things work while at the same time being adept at driving outcome and impact. I am passionate about helping motivated early-career technology workers achieve their potential.
$ 199 ⏰ 45 minutes

Get access to Chen Zhao's calendar after Stripe process your payment, fully refundable if not 100% satisfied.

⏰ A/B Testing with Chen Zhao for $199

  • I'm a veteran software architect with 15 years of engineering experience in the area of computational advertising, big data analytics, and distributed ML training, currently working on a next-generation platform for distributed training pipelines at Facebook
$ 199 ⏰ 1 hour

Get access to Chunzhao Zheng's calendar after Stripe process your payment, fully refundable if not 100% satisfied.

⏰ A/B Testing with Chunzhao Zheng for $199

  • Alex is a Principal Product Manager at Sambanova Systems. Previously, Alex has held product management roles at C3.ai building out their ML Platform and deep learning offering to enterprise customers. He was also an ML practitioner as the first data science hire at Uber’s Advanced Technologies Group working on building large-scale computer vision and sensor fusion algorithms for self-driving cars. Alex is motivated to help raise the next generation of data scientists and product leaders
$ 199 ⏰ 45 minutes

Get access to Alex Chao's calendar after Stripe process your payment, fully refundable if not 100% satisfied.

⏰ A/B Testing with Alex Chao for $199

  • I am Tech Lead/Product Manager at Alexa @ Amazon. I have been working at Amazon for past 9+ years and have experience in Front-End, Back-End tech frameworks. I am well versed business courses like Finance, Operations, Marketing, Product Management, Macro/Micro Economics etc.

“I have done this for 9+ years and have helped many people with what they wanted to achieve, that is why I am so good at this.”


$ 199 ⏰ 1 hour

Get access to Aniket Phatak's calendar after Stripe process your payment, fully refundable if not 100% satisfied.

⏰ A/B Testing with Aniket Phatak for $199

  • 16+ years of experience in product and strategy in multiple tech and consulting firms
$ 199 ⏰ 45 minutes

Get access to Jun Oh's calendar after Stripe process your payment, fully refundable if not 100% satisfied.

⏰ A/B Testing with Jun Oh for $199

The easiest way to accomplish something is to hire someone who's done it

Selected Customers Testimonials

50% Salary Increase!

Thank you so much @Leon !

Your advice and coaching are extremely helpful for preparations for the interviews. The timeline and pacing you helped plan worked out very well.

It kept me on the right track. The exercise on coding and data analytics you recommended greatly sharpened my skills to solve the problems during the technical rounds.

The practice to communicate project work to peers is another highlight to help me earn higher score. Again thank you so much for all your advice!


instamentor | customer review
Joey Huang

Machine Learning Engineer

Dream Job Offer from Apple!

Hi @Leon and @Mike ,

Just wanted to share, I accepted an offer from Apple as a Data Engineer and wanted to offer a huge thank you to both of you guys.

Even though I only shared a small time with you both I benefited a lot from your platform and your mock interviews.

Some of the questions you gave during our Data Modeling with you Mike were presented verbatim.


Steve

Data Engineer at Apple

The mock interview helped me do it in the right way

I did a SQL Mock interview on Instamentor to gain a better understanding of how SQL interviews are conducted.

Leon (my mentor) came prepared with a realistic case study example tailored specifically towards the company I was interested in. Read more...

The questions and tips he provided were incredibly helpful in my preparation.

My actual interview went in a similar manner and having done the mock interview helped me do it in the right way.


instamentor | customer review
Nisheeth

Data Engineer at Square

Extremely valuable!

Shoutout to Leon Wei from instamentor.com.

Please do reach out to him if you're looking for career advice, general consulting, or resume feedback. I found his services extremely valuable.

And while you're at it, please do check out SQLPad if you're looking to refresh your SQL concepts.


instamentor | customer review
Jonathan D'Cruz

Data Science Manager at Swiggy

Recommending him!

Shout out to Leon Wei from instamentor.com, recommending him if you are searching for great advises and mentorship for your #datasciences career.


instamentor | customer review
Feng Xue

Senior Data Engineer at T-Mobile

Dream Data Scientist job offer!

I've tried several sites to practice SQL and sqlpad.io is my favorite because the questions are well-designed to focus on the most important concepts. Some of the other sites are all over the place.

With sqlpad.io, I can focus on questions by skill level or topic, and the number of questions available is a reasonable amount to study for interview prep.

In addition, I got mentoring sessions with Leon and I was impressed at how helpful it was. Leon was calm and knowledgeable, which helped my nerves over my up-coming interview.

He really takes mentoring seriously and it makes a big difference. After practicing my SQL, I got my data science dream job!


Jennifer

Data Scientist

Meticulously prepared and very professional

I signed up on Leon's service and was pleasantly surprised when he helped me out personally on the website. In two mock interviews with him from instamentor.com, he was meticulously prepared and very professional.

Throughout the interview he provided fantastic feedback and frameworks on how to be better prepared for both technical and behavioural aspects of an interview.

He is also a very positive and friendly person and I enjoyed all my interactions with him.


instamentor | customer review
Rahul Nayak

Marketing Analyst

Thank you!

Shoutout to Leon Wei from instamentor.com, recommending him if you are searching for great advices and mentorship for you #datascience career.


instamentor | customer review
Bara Ndiaye

Data Scientist

What made the session successful was the effortlessness in being honest around Leon.

I had my first mentoring call yesterday. The session was to be for 45 mins but lucky for me, Leon did not have a hard stop. We had a detailed discussion around my career history, career transition, stuff I’m good at and enjoy doing/good at but don’t enjoy/not good at and want to improve, not good at and not interested in improving...you get the drift). Read more

Leon listened patiently, highlighted areas I needed to work on, pointed out practical considerations, and made gentle suggestions on handling a couple of situations I had bombed (salary negotiation was one).

We covered a lot of ground and had concrete to-dos at the end of the call, but what made the session successful was the effortlessness in being honest around Leon. A mentor needs to identify what is holding the mentee back in their career, and their advice can only be as good as the inputs we give them. Leon made it easy to be authentic. I’ll keep you all posted on my journey!


A Mentee

Data Scientist

I landed the job I wanted at a FAANG company

Joining INSTAMENTOR.COM was very easy, and I quickly found a mentor who is a good match. My mentor was amazing and friendly. He knew all the details and important things that I needed in my job search. I landed the job I wanted at a FAANG company, and we’re still in touch.


Jason

Data Scientist, received a dream offer

Instamentor is understanding and accommodating

Although my first mentor was great, I decided to pursue a different career track mid-way. They quickly found me a different mentor after I contacted them, without extra charge.


Diego

Recent Graduate

This platform and my mentor have made the job search so much easier

This platform and my mentor have made the job search so much easier for me. All the support and insights from my mentor made me feel so much readier for my interview. This made me walk in completely relaxed and natural in the interview room which got me the job!


Sandeep

Data Engineer

Instamentor helped me connect with a mentor from my dream company

Instamentor helped me connect with a mentor from my dream company. I learned a lot about the culture of the company, the good and bad about working there. He also help me prepare my job interviews by providing mock interviews and resume feedbacks.


Alex

Software Engineer

Instamentor hooked me up with a mentor who changed my career

I’m a business analyst and always want to become a data scientist. I’m really glad Instamentor hook me up with a mentor who successfully changed my career path. With my mentor’s guidance, I’m on track learning coding languages that are crucial as a data scientist. The most important thing is that I have gained courage and confidence on my career development.


Su

Data Analyst

BOOK YOUR SESSION