Our algorithms have sex
This is the fifth in a series of interviews with members of the Machine Commons supplier Collective. Subscribe to the site to be alerted about future posts, or become a partner today!
Ex-Googler Gabriel Hughes is the CEO and co-founder of Metageni, an AI powered analytics and web optimization firm.
His story is noteworthy because he is the father of modern attribution modelling (true story, see original Google patent) and in December the company was awarded Best Use Of AI in the London Ecommerce Awards, for their work using genetic algorithms to optimize conversion rates.
Gabe seems happy enough despite the pandemic, acknowledging both the polarized views of the public about the various covid-19 restrictions and the fact his own life has been relatively stable compared to some people.
For many working in the machine learning space, life is remarkably normal. We’re all used to working in isolation, from our computers. The trend was already in motion well before masks became a necessary ticket to the outside world.
How’s business been?
“We had three travel clients. We did really well with them for a while, then one particularly successful client pulled out when covid hit… those travel clients didn’t come back again in 2020!”
“It’s been really interesting. A bit of a roller coaster ride. There’s great demand for expert data analytics and we’ve always had an online focus. Ecommerce is looking pretty good – we’ve seen an increase in revenue from these kind of companies.”
“Occasionally it’s difficult to explain ‘AI’ in a way people understand, that machine learning isn’t just a robot thing!”
How did business change after the pandemic?
“The common thread across travel and ecommerce companies is to improve online sales with paid advertising, growing costs. A lot of companies are now experimenting more with SEO (search) and social.”
“I've done this almost my whole career. Analysing the customer journey through to purchase.”
Very hard though, isn’t it, who knows what goes on under the hood with search!
“Well, when I worked at Google, I patented the original attribution modelling in Google Analytics. It then rolled out across G-suite.”
Oh. Right then.
“There’s a few patents I’m associated with from my Google days – multi touch attribution and the attribution modelling tool was the most significant.”
US8788339B2 - Multiple attribution models with return on ad spend - Google Patents
What do you do?
“At Metageni, we’re trying to create objectivity using data based attribution to try and suggest one attribution model is more correct than another.”
“The purpose is to allocate the value of the sale back to the marketing channel so we can better identify ROI of each one.”
“So this is what we did with travel clients, we used customer journey data to compare buyers to the people who don't buy, compare the conversion and identify the contribution of each channel.”
“This identifies each feature’s contributions to each channel, to model what features predict user conversion. In this way, we’re able to allocate budget to those things that predict conversion, therefore increasing ROI of the overall budget.”
“That's the whole raison d'etre of attribution.”
What makes one model patentable?
“Well, I'm not a patent lawyer! …We were running prototypes and the product team started getting interested. Then you have to start thinking about IP! They kind of did it all.”
“In my current role when we last looked into this we got the advice that you can't actually patent an algorithm in the UK and Euopean markets. I believe the law is different in the US compared to the rest of the world.”
How so?
“I don't know. I think in order for it to be patentable you have to tie the algorithm to something that actually does something. Something that makes an automated decision – as opposed to a human interpreting analytical output then manually changing spend. ”
How is e-commerce compared to travel?
“Any client who sold something online and needed marketing efficiency (data driven) was potentially interested in us. Not just for digital marketing spend, but for conversions and personalization.”
“Clients were interested if machine learning algorithms could optimize conversion rates in real time.”
How’s that new?
“ML personalization is when you predict customer purchase likelihood and use this to introduce interventions on the website, responding in real time to customer behaviour.”
“The output is different. It’s not a recommendation to spend more or less money on a certain channel but a dynamic change to consumer facing elements.”
Interesting, like a recommender system across more of the customer journey.
“Something like that yes – we cannot go into too much detail for client commercial reasons but we are very proud of the award and are continuing to work with AO.com who have been working very hard to evolve and optimize their ecommerce during lockdown”
Award winners! - Metageni with AO.com win 'Best Use Of AI' - Metageni
He tells me that some of this work can be applied to other client situations, whilst some of it remains privy to AO.
What’s the technology, the key innovation?
“It uses combinations of ensemble and some neural network (not all deep learning actually) – whatever model is appropriate to the problem, we like simple decision trees because they're transparent and interpretable.”
“Something new I picked up was around feature selection and model optimisation using genetic approaches.”
“This way, you’re able to explore lots of different models. There’s a search space out there and when you realise how big it is – even for a simple problem, there are more possible models than with best computers that you would possibly have time to explore.
So something like a decision tree, which does not perform as well as an ensemble or deep learning approach, can be improved on by searching for a better feature and parameter combination.”
Evolutionary machine learning meets marketing analytics - Metageni
What are genetic algorithms?!
“Genetic algorithms are really cool. Like in early days of ML, they've made a bit of a comeback. Certain classes with huge search spaces, where you have to try lots of different alternatives, you create a population of candidate solutions.”
“You evaluate them all against objective criteria, then you pick the winning ones, mutate them slightly and create a new population by cross fertilizing bits of solution to create a new population solution.”
“So. As I like to say: our algorithms, ummmm, have sex.”
“You create lots of different versions of the model, basically finding one that wins.”
“The school timetable is a good example: what's the optimal time table that minimises clashes – what’s the best timetable? You can randomly generate a bunch of timetables then assess against criteria and use that to generate a new sample.”
“Then after a few cycles you end up with something that's like whoa. It won't guarantee you find the global optimum but it's a good way of avoiding zoning in on a local optimum.”
“You do need to be careful in a few areas. One big 'gotcha' is you need to avoid unwittingly training your models on hold-out data using the genetic objective function as a kind of back door signal into the models – but once you have the method set up it's very powerful.”
(FYI Data leakage (‘a back door signal’) is when the data scientist unwittingly allows a model to train on data you’re not supposed to have access to; the ‘genetic objective function’ is the variable you’re prioritizing, assessing the randomly generated attempts against.)
“For the most part, the ML is fairly straight forward. It's really about good feature selection.”
What’s next with Metageni?
“With Metageni, I'm sort of still trying to figure out how to leverage neural networks beyond what they already do really well. So, NLP or image recognition. Something that's complex and unstructured would be great. As a company we need to speed up feature extraction so I want to find ways to automate that. We also need to keep focused on making it relevant for the clients.”
“I wish we had the luxury of doing loads of R&D, it's currently all about… well, R&D doesn't earn you money: you need to pilot tech and run with it to make you money. ”
It’s a shame pragmatism and idealism aren’t better aligned.
What do you think the future holds for machine learning?
“In terms of R&D and where it's going, it's about finding models that can adapt over a longer period of time and don't require constant re-training.”
“There's work that's being done on this, work that makes ML more generalisable.”
“Also, the data gap: how do you [automatically] get from unstructured data to data that can be used for the purposes of training a model?”
What are you excited about for the future of machine learning?
“In truth I'd like to be able to operate in a world where you're not always using AWS, Azure, Google Big Query; anyone who's doing stuff at scale with cloud data is generally using one of those three stacks.”
“I mean, it's fine. The toolkits these guys are empowering is incredible. But it feels a little bit like they want to just do it all for you and we're reduced to dumb participants. We need more people who can pick it up and run with it with a sound understanding of what is going on under the bonnet.”
“If you have that expertise, your main limitation is just data. It's about how you represent the world in data.”
“It still hasn't happened yet. Everyone is aware of this… we’re on the brink of something much bigger…”
I’m tingling.
What do you mean on the brink of something much bigger?
“Well most of the big changes will be for quite boring applications! I have friend in a major law firm, he's talking about simple stuff like screening claims and applications, but lots of applications like that would add up to have huge implications for all types of organization and for society as a whole.”
My tingling abates slightly.
“Anything where you have to repeatedly think a problem through based on information, is vulnerable to ML automation.”
“The problem right now is that you have to build those systems in such a way – big training set, etc. – that you’re only able to answer a narrow field of questions [that ‘look like’ the training data].”
“Only when you generalise data to more general problems that the really big leaps will come.”
Thoughts on artificial general intelligence? Are we doomed?
“Yeah I've read the scary books… Superintelligence… It's all over for human kind…”
"I'm sort of a bit optimistic about it.”
"I can't help but think intelligence - if it's real intelligence - will tend to enlightenment and wisdom. Intelligence is associated with morally correct behaviour.”
Now that’s a beautiful view.
“Maybe I'll regret it when the robots take over!”