Blogging is a conversation.
I’d love to hear your thoughts!
Cambridge Centre for AI in Medicine Summer School
Last week I had the opportunity to join the Cambridge Centre for AI in Medicine Summer School that provided a deep dive into the future of machine learning in healthcare. Over the course of five days we had world-leading experts in the fields of AI, machine learning, and healthcare covering topics including clinical trials, treatment effect estimation, quantitative epistemology, causal deep learning, AutoML, graph neural nets and applications in genomics, risk prediction, survival analysis, longitudinal studies and synthetic data.Below are some of my key takeaways from a product perspective.
While there have been significant new developments in machine learning for healthcare, there is a gap from potential to translation. Some ways to address this gap:
Choosing the right problems with clinical relevance and appropriate data
Considering the ethical implications and potential biases upfront
Rigorous evaluation and thoughtful reporting of model performance
Deploying responsibly and providing safety monitoring and model updates
Reproducibility continues to pose a challenge. Being able to replicate results particularly in healthcare is essential for ensuring that the model is credible and reliable. In an attempt to solve for this, NeurIPS, a premier artificial intelligence conference, now asks researchers to submit a “reproducibility checklist” including items often omitted from papers, like the number of models trained before the “best” one was selected, the computing power used, and links to code and datasets.
FDA is continuously updating guidance on good machine learning practices. You can read more about FDA guidance on machine learning for software as a medical device here.
It is important to address data access and security vulnerabilities. These applications of federated learning and swarm learning appear promising.
There is a need to explain even highly complex models to verify predictions, identify flaws and biases, learn about the problem and ensure compliance to legislation. EU GDPR articles 13-15 provide rights to ‘meaningful information about the logic involved’ in automated decisions. Two approaches:
Interpretability by Design. Simplify the ML model.
Post-hoc Interpretability. Create a “language” between opaque models and humans.
AutoML may help to increase ML adoption in healthcare through easy deployment of up to data ML methods. A case study from Google Cloud here.
Using multiple studies can help enhance precision and increase generalizability however it poses challenges such as harmonization of information, accounting for measurement errors and missing data.
Clinicians want to see the stage, state and progression of disease. Accurately forecasting individual-level disease trajectories can enable early treatment.
An inspiring case study by AWS of using computer vision analysis to create a faster, less expensive, more reliable, and more accessible system to screen children early for Autism Spectrum Disorder.
Lack of high quality healthcare data impedes ML research in healthcare. Synthetic data may help address this.
Clinical trials are difficult, expensive and ripe for disruption. Half of randomized trials exclude 77% of people they are trying to treat and we then widely extrapolate from obtained results which means that patients who are receiving the drug might have been excluded in the clinical trials for the same drug. We need to ensure that machine learning for healthcare works for all.
The summer school was a fantastic learning experience and I feel excited by the opportunities for product thinking in machine learning for healthcare.
3 Day Entrepreneur First Taster
I believe that product management is one of the most entrepreneurial roles in tech and there are several overlapping skills that make successful product managers as well as entrepreneurs. Whether you’re part of a large company or a new start-up venture, you often need to think like an entrepreneur if you manage products. And quite often, entrepreneurs later turn to product management careers. For this reason, I love learning from entrepreneurship focussed resources and hanging out with entrepreneurs.
I had reached the halfway point in 2022 without any holiday so I decided to take a week off for a stay-cation in July and also spend some time on personal development outside of my job. I came across a 3 day programme with Entrepreneur First (EF) that enables you to discover what it’s like to be a founder and build, ideate, and collaborate in person. I was fortunate to be among the ~50 candidates that got selected out of 600 people that applied.
What is EF?
EF selects individuals they recognise to have founder potential and bring them together to find their co-founders. They create the ideal environment for them to trial partnerships, and test ideas at a rapid pace. They give them access to a global network of founders, advisors and investors. They even cover their living costs while they are on the programme.
I was very impressed with how well the 3 day taster programme was organised. The communication before the programme even started was fantastic. They shared pre-work and reading material to ensure that participants are prepared to get the most out of the 3 days, created groups as the one-stop for updates on the operations and logistics for each day, and also asked for everyone to create a profile in to introduce themselves, let others know what they are bringing to the programme and start connecting with others.
Image source: https://www.joinef.com/
Without giving too much of the programme away, the 3 days essentially consisted of various talks, panels, ideations sessions and presentations to give you a flavour of what the full EF cohort can offer. There were recent graduates of the full cohort speaking to us about their experience within the programme, creating their start ups from scratch, getting funding and growing their companies beyond EF.
EF has developed their own way to help founders identify the problems they are best suited to solve. It’s called Edge.
What is Edge?
An individual’s Edge is their unfair advantage in solving a problem, compared to other founders. Each individual who joins EF is encouraged to reflect and to think about their unique skills and experiences. EF then helps them think about how these could give them an unfair advantage within a particular market.
We spent some time define our edge/edges based on questions such as below:
What are the things you know about that few others know?
What can you build that few others can?
What unique or rare insights do you have that others don’t?
We had some speed networking exercises where we shared our edges with the other participants and discussed any strong convictions and beliefs about the future that we shared that could help us come up with a start up idea together.
EF then matched participants based on complimenting CEO/CTO edges for 2 ideation sessions and we were able to pick our own match for a 3rd ideation session. One of my fears going into the EF taster was the idea of a potential co-founder ‘break up’ but I was amazed by the safe space they created to discover if a potential co-founder is not a good match and to cut your losses earlier.
The 3 days flew by and I walked away feeling incredibly energised having spent time with some of the most ambitious people I had ever met, thinking big to solve real world problems and making new likeminded friends. Following the programme, I was among a handful of people to be offered a fast-track to the full cohort but I will be holding off on full time entrepreneurship while I am getting a lot of fulfilment from my product management career path. The learnings from the programme however have been invaluable in my job and I recommend any product managers to check out Entrepreneur First’s reading list to maximise their impact.
Lessons from Product Courses - SVPG Inspired Workshop
Last week I participated in Silicon Valley Product Group’s INSPIRED: How to Create Tech Products Customers Love workshop that was conducted across 4 days via Zoom in four, 3.5-hour sessions. We went through best practices, practical examples and case studies related to product discovery, testing, setting objectives, data analytics and stakeholder partnership.
Here’s some of my key takeaways:
Address product risks early - is this product valuable to users, usable by users, feasible to build and viable from a business perspective?
We need to test feasibility and viability during discovery not after.
We can’t count on our customers to tell us what to build, most of the time the customer does not know what is possible. We invent on behalf of our customers.
If you don’t get response to your user testing recruitment, it might not be a big enough problem to solve.
We expect that most of our ideas won't work and those that do require several iterations.
Product vision is one of the most critical artefacts, everyone needs to know what the product vision is 3-5 years out.
We create MVP Tests (typically measured in days) in order to discover our way to Product Market Fit (usually takes months).
Customer development programmes/discovery partners can be helpful to reach product market fit.
Our goal is to test out ideas as quickly and cheaply as possible e.g. wizard of oz testing, PR FAQ, fake door test, explainer video.
Do an opportunity assessment for every single thing going into the backlog: What are we solving? How we know it really is an opportunity? Who are we solving for? What business objective are we focused on? How do we know that we have succeeded?
Don’t ship your org chart.
Focus on impact and leading metrics - leading indicators are making the news, lagging indicators are reporting on the news.
When showing stakeholders low fidelity prototypes, tell them the user story which is exciting. Start presentations with demo/story telling.
Data beats opinions. Try to speak in the customer’s voice rather than your opinion.
Build outcome-based roadmaps. Change the conversation from features to results.
Workshop link (thoroughly recommend): https://www.svpg.com/inspired-workshop/
Modernisation of 1000+ websites with zero downtime using Google Cloud
One of the most exciting opportunities I had in my cloud platform engineering role was managing the migration and modernisation of 1000+ websites with zero downtime using Google Cloud Platform.
While many of these websites were already hosted in another cloud provider, they did not yet realise the full benefits of cloud. The websites had been lifted from on-premises infrastructure and had not been modernised to make use of cloud and DevOps best practises, making them difficult to operate and maintain. Our platform team embarked upon an effort to modernise these websites using Google Cloud, with the goals of increasing efficiency, improving security posture and reducing operational toil.
Dedicated platform engineering
Key to the success of the website modernisation effort was our dedicated platform engineering team. The platform engineering team was a group of cloud and DevOps experts with a passion for automation and modern software practises. The platform team’s vision was to empower other engineering teams in the company to quickly and safely develop and operate applications on Google Cloud.
To help realise this vision, the team built and operated a self-service platform that other teams could use, providing functionality such as:
Automated Google Cloud project creation and billing setup, applying principles of least privilege
Automated infrastructure provisioning using Cloud Build for CI/CD and Terraform for infrastructure-as-code
Reusable Terraform modules that create and configure Google Cloud resources according to best practises, helping teams get started quickly and safely
A GitOps-style peer review process for infrastructure changes
Automatic enrollment in Security Command Center, providing threat detection and web analytics
Automated response to security incidents like misconfigurations, using serverless tools like Cloud Functions to respond to events
Automated security and compliance testing
Centralised log collection and monitoring
Modernising Internet Hosting
The web estate to be migrated was complex. The websites used a range of different technologies and platforms, resulting in high operational burden, high total cost of ownership, and increased security risk. The internet hosting team wanted to modernise this web estate in order to reduce these burdens and increase efficiencies.
The internet hosting team approached our platform engineering team to explore how they could collaborate on the website modernisation projects. We worked together to define a structured approach for the website modernisation. First, we ran deep-dive sessions to educate the hosting teams about Google Cloud. Next, we onboarded the hosting teams onto our internal platform for creating and operating best practice Google Cloud projects. Working alongside Google Cloud Professional Services, we mapped current infrastructure to Google Cloud, highlighting areas for improvement and modernisation. We built the infrastructure in sprints, showing incremental value along the way. We adopted a phased approach, starting with lower traffic websites first, and building up to the most valuable and high traffic sites.
The internet hosting team adopted several key Google Cloud technologies to modernise this web estate, including:
Cloud Load Balancing and Cloud CDN to serve our global customer base
Cloud Armor to protect the websites against denial of service and web attacks
Managed Cloud SQL databases to reduce the operational burden of managing their own databases
Shared VPC, so that the network team can administer the networking configuration of the internet hosting projects
Compute Engine managed instance groups to meet our scalability and high-availability requirements
Results
The team successfully migrated over 1400 websites to Google Cloud. The migration was achieved with zero downtime so our customers and businesses were not disrupted. Overall performance of the web estate has improved, including a significant reduction in site response times.
Using the tools and best practises created by the platform engineering team, the internet hosting team has achieved significant long-term benefits such as:
Reduced total cost of ownership through automation and standardisation of the web estate
Improved resilience and reliability through automated infrastructure provisioning, CI/CD and infrastructure as code
Improved security posture by automatically applying principles of least privilege and adopting DevSecOps best practises
Improved incident management through more effective telemetry, monitoring, and alerting
Better cost management by automatically creating budgets and labels
Crucially, the internet hosting team now independently operates its Google Cloud projects and resources in a self-sufficient manner, with limited support from our platform engineering team. This enables our platform engineering team to scale our efforts and empower other engineering teams to modernise their infrastructure and workflows using Google Cloud.
How I Passed the Google Cloud Associate Cloud Engineer Exam
I joined GSK on the Tech Future Leaders programme after my degree in Biomedical Engineering. My first role was as a business analyst, and second as a project manager, both leadership and management focused. I grew immensely from the experience but found that despite my biomedical engineering background, my confidence would get a bit shaky when navigating technical software conversations. I was keen to pursue a career in product management and felt that building my computer science expertise further will help my career in the long term.
During my time at Galvani Bioelectronics I started getting more involved in and enjoying hands on technical tasks. After getting some exposure to Google Cloud Platform (GCP), I decided to get more serious about building up my cloud engineering skills in time for the end of my grad scheme.
As a goal oriented learner, I decided to book myself onto the Google Associate Cloud Engineer certification exam which focuses on one’s ability to plan, configure and deploy cloud solutions using various GCP services and features. Having never been in a technical role before, and university times far behind, I knew I had to relearn learning. I determined that even if I didn’t pass the exam, the knowledge I gain from the preparation will be enough to make it worth the effort. An exam fast approaching was all the motivation I needed to get a study plan together. There were plenty of resources available online but it was difficult to figure out where to start. I enrolled onto the Google Cloud Engineer modules on Coursera which also gave me free access to Qwiklabs (self-paced labs that help you learn how to work in GCP through step-by-step instructions). I also signed up for a free trial of GCP so I could build in my own environment as I went along.
Although the materials on Coursera and Qwiklabs were fantastic, it was challenging to find time and the right head space in the work day to dedicate to learning. Luckily (or not), I had a 1.5 hour commute each way that I began to use to go through the online courses. I was making progress but not fast enough to be ready for the exam 1 month away and I had a piling list of questions from all the self-study. I found the answer to my woes in an instructor led 3 day GCP Infrastructure course by Jellyfish Training. Not only was I able to get all my questions answered, I also got a chance to do more hands on tasks in GCP, reinforce my online learnings and review many more practice questions.
The run up to the exam was still quite hectic as I was new to so many concepts that I hadn’t encountered in practice before. I also ended up taking a couple days of annual leave which maybe in hindsight was overkill but gave me peace of mind that I had given it a good shot. And voila! I passed!
Passing this exam was a huge confidence boost and helped me land a role as a Platform Engineer. It was by no means a sail into the sunset from here as learning never stops and is even more so the case in technical roles. I was challenged every day by the automation, security and compliance considerations when building cloud technologies and also worked towards the Professional Cloud Architect exam (future blog). The learnings from the Associate Cloud Engineer certification were minuscule compared to the experience I gained in this role but it gave me the confidence to take a leap of faith into a completely new area for which I am very grateful.
