Eric Glover Ph.D.
E-mail: eric_rez8@ericglover.com, Web: http://www.ericglover.com/, LinkedIn: https://www.linkedin.com/in/erglover/
__________________________________________________________________
About:
A strong technical leader and seasoned AI/ML/DS/LLM professional specializing in taking search and generative AI-related and mobile products from 0 to 1 – with broad expertise in the developing search and LLM/Chat products for mobile and web-based (multi-lingual) search and advertising, general document classification and extraction, large- and small-scale machine learning, knowledge-graphs and a strong decision-support and business-value-driven philosophy.
Highlights:
• Founded AppliedIngenuity.AI, a pioneering consulting firm and blog, committed to demystifying cutting-edge AI technology. Our mission is centered on equipping businesses with the tools and knowledge to effectively leverage AI/ML in their user-facing products, thus maximizing their potential and enhancing user experience.
• Expertise in LLMs and ChatGPT through https://appliedingenuity.substack.com/ where I publish novel works related to LLMs and ChatGPT, as well as overview complex technology to provide intuition to average technical people.
• Co-founded and serve as CTO at Sage.guide, where I spearheaded the development and implementation of a personalized benefits search and support tool. This innovative platform harnesses custom-built domain-specific semantic search and knowledge graphs, coupled with the power of OpenAI’s APIs, including Fine-tuning, to optimize concept extraction. Our system can proficiently interpret and respond to queries related to both tabular and textual data, while providing robust safeguards against potential LLM risks such as hallucinations or bias.
• Advised on creation of what became an FDA authorized Machine Learned algorithm to predict if a young child had autism, or needed further evaluation (Cognoa)
• Key designer and implementer of a Natural Language Processing Engine for a mobile search product covering parsing, understanding, ElasticSearch query generation (Branch)
• Designed and implemented core enhancement to multi-Billion user-identity graph which probabilistically predicts multiple identifiers from the same user through scanning > 1 TB of events daily (Branch)
• Improved ultra-short query remote-powered app-search CTR by 3X by developing a (multi-locale – language, region) predictive model of intended app for queries as small as one character (Branch)
• Led team to design and deploy (> 50M devices) an on-device personalized app-search algorithm leveraging both on-device (private) and server-synced data to identify and rank app-results. Ultimately led to multiple OEMs and Carriers incorporating our SDK – hundreds of Millions/Billions QPD.
• Led small team to build very high-performance Chinese Language Tokenizer, outperformed Alibaba’s. (Quixey)
• Designed (implemented Python version, drove JAVA engineering) microservice that could predict in under 100 microseconds if our NLP system could answer a query. Was limited-deployed with Ask.com mobile for monetization in five verticals.
• Defined DCG guidelines and assisted in the design of the Quixey Relevance Testing system – the basis of our MLR training and evaluation (Quixey)
• Designed and implemented a large-scale system for non-engineers to execute and manage automated web-scale categorization using active learning – ran on Billions of web pages and covered more than 1000 categories. (SearchMe)
• Designed and lead implementation of very high-performance query-based personname detection engine (< 10 microseconds in 2006). (ASK.com)
• Designed and drove engineering effort of an automated smart-answer system based on Wikipedia data that could handle synonyms – covered more than 15% of Ask.com traffic at peak.
Experience:
Machine Learning Specialist June 2023 – Present
Chip Scan is a company in the hardware security space, pioneering the use of Zero Trust techniques for Hardware, assuming that no component or user can be trusted until proven otherwise. We specialize in understanding hardware in ways different from everyone else. My job as a machine learning specialist is to apply my expertise in hardware, software, and AI/ML, especially LLMs to improve the understanding of any given piece of hardware. I am responsible for our LLM running in our private cloud, as well as the surrounding systems used to improve human understandability of hardware.
Technologies used:
- LLM/Llama-2/Custom models/ChatGPT/Prompt Engineering
- Private cloud GPUs
- Distributed ML
- Verilog
- Python
- Deep Learning
Remote Creator/Content April 2023 – Present
Created AppliedIngenuity.AI a blog/consulting firm dedicated to making the latest AI accessible to all with a focus on maximizing Business’ ability to effectively utilize AI/MI for user-facing products. Specializing in LLMs, Prompt Engineering, Search, and the latest AI topics.
CTO/Co-founder Sep 2022 – June 2023
Sage.guide is an all-benefits employee guidance and employer decision-support company. As a co-founder and CTO, I am responsible for the technology vision – how to improve the employee experience related to documents, services, and benefit through an AI (LLM + knowledge-graph)based platform. I designed, and drove the implementation with our remote team, of a flexible, dynamic, knowledge-graph-based platform that supported benefit search and personalized recommendation ranking of employee benefit suggestions.
Technologies used:
- LLM/ChatGPT/Prompt Engineering
- OpenAI API including fine-tuning
- Document/concept embeddings (local LLM – BERT)
- Semantic/vector search
- Key-phrase/concept extraction – domain-specific fine-tuned model
- Knowledge Graphs – custom designed for the employee benefits domain
- Python, AWS, MongoDB
Our first-generation system, built in weeks, was able to trivially enter/encode benefit knowledge such as eligibility, savings calculations, and interactive dynamic UI elements. The initial demo won 5th for the day on Product Hunt.
CTO Sep 2022 – May 2023
Finny, sold to Origin, is a financial wellness platform, empowering employees to make better and more confident money decisions. We work with employers to solve the #1 stress employees bring to work: their finances.
Principal Data Scientist July 2016 – July 2022
As Principal Data Scientist, I was involved with several different parts of the business. My primary responsibilities were to support other data scientists and engineers as well as design and implement to POC novel algorithms or solutions in areas related to attribution, and mobile search and discovery. I created the Data Science Guild and ran the first Data Science Meetup (https://www.youtube.com/watch?v=rQAL02Hdkws) as well as advised and mentored multiple data scientists.
Highlights:
• Designed, implemented, deployed, and patented the first probabilistic matching produced a true probability score. Built and deployed years before Apple’s ITP and IOS 14.
• Designed and implemented core logic of a multi-vertical natural language mobile search system which powered Billions of queries, including on Samsung S10 devices. System could answer queries like: "things to do", "vegetarian near me", "New York Pizza", "Portland Hardware Stores", “311” (the band) with ability to detect and resolve geographic, entity and vertical ambiguity.
• Improved CTR of remote mobile app search by 3X by explicitly predicting probabilities of intended apps, with more than 75% accuracy for single character queries, > 95% for 3+. Effective for multiple regions and languages.
• Led a small team of engineers and data scientists to create the first on-device personalized, real-time, app-search algorithm (SQLite query). Production live with over 50M devices. Algorithm leveraged both local (private) and remote-synced data, and could handle typos/alternatives, with extremely high accuracy for even one character.
• On first and second place teams in company pitch day. Led 2nd place team by proposing shifting to a more modular - use-case driven business model. Within 6 months, parts of pitch have begun to bear fruit throughout the org.
• Advised on the Media Mix Modeling project (MMM) - Assisted in developing business materials, direction and modeling to predict ROI for each channel without individual attribution.
• Created Branch Data Science Guild – focused on educating everyone (esp. non-DS) about data science/machine learning
• Facilitated and participated in the first Data Science Meetup
(https://www.youtube.com/watch?v=rQAL02Hdkws)
• Advised and mentored multiple data scientists
Cognoa Inc Palo Alto, CA
Adviser Aug 2016 –2021
Cognoa is a behavioral health company that develops digital diagnostic and therapeutic products with the goal of enabling earlier and more equitable care for behavioral health conditions.
As an adviser for Cognoa, I assisted in the early work on what would later become their FDA Authorized algorithm for predicting childhood Autism. I advised on improving model accuracy by creating a third class of result corresponding to uncertain or recommend further medical evaluation. I also assisted and advised on techniques for being able to create effective models with extremely small initial labeled data – working directly with their Chief AI officer. This work resulted in multiple medical-journal publications.
Quixey Mountain View, CA
Engineering Fellow : Aug 2011 – May 2016
Quixey is an app-search and discovery platform company. Known for functional search and deep-app search and app-mining. With major funding from Alibaba and a strong presence in the Chinese market.
_ Founding advisor – advised the founders, convincing them they could build app search _ Responsible for early development of ML relevance model for app-search
_ Developed app search in English, Korean, Spanish, and assisted in Chinese App Search
_ Developed app-to-app similarity technology, launched by Microsoft to convert users from Android/IOS
_ Co-developing early prototype deep-app content search
_ Numerous patents (see below)
_ Developed (and prototyped in Python) knowledge-based query routing engine (sub 1 ms)
_ Designed and lead development of very high-performance knowledge-based query parsing tech < 20 microseconds per query
_ Designed and implemented automated content aggregation system
CEO and Co-Founder Sep 2009 - Aug 2011
Intelligent Search Solutions was a small angel funded media search company with an interactive Flash-based UI
_ Designed and mostly implemented back-end search and personalization engine o Including autosuggest, search, ranking, autocorrect (domain-specific) _ Designed large-scale crawling system that refreshed over 1M videos per day
SearchMe Mountain View, CA
Principal Scientist/Classification Architect Mar 2007 – July 2009
SearchMe was a visual multi-media search engine that had its own crawler and autocategorization technology to real-time disambiguate search queries into over 1000 categories.
_ Web-scale categorization system (CHOCO): Technical lead/primary coder - large-scale categorization system (more than 1000 categories) - run on Billions of documents in days. System included full web-based UI for training, active learning, evaluation, ontology management and automated error checking to aid Search Analysis/contractors.
_ Vertical suggest/query intention mining: Leveraging full document to category matrix, produced a ranked list of "vertical suggestions" in real-time per search.
_ Multimedia blending: Technical lead - project ranked and index multimedia (YouTube, Hulu, Imeem, Flickr and others) content. Solved complex AI problems related to different features from text-web pages. Implemented custom automated feed processing and defined techniques for efficiently discovering appropriate pages to index.
_ Competitive analysis/judgment collection system (TORGO): Technical lead - Internal web-based system for collecting judgments for competitive analysis and MLR training. System included scrapers to pull data from our competitors and caching system, easy to use UI. Flexible design of code and DB schema to rapidly change user-judgment options.
_ MLR feature design and development: Various roles - coding, system design, and formally defining features. Key accomplishments include defining and design specification of XPATH/Perl based system for rapidly (hours) adding totally new features including use of structured data.
_ Near-real time feed processing system ("El Rapido"): Technical lead/primary coder - near-real-time data inclusion from RSS feeds. System included full process from fetching and parsing feeds to indexing and generation of MLR features. Search Quality team managed feeds and options (via Excel). Significant perceived relevance boost - time from initial proposal to live under 3 weeks.
_ Lead projects on automatic page quality, spam, and many relevance (DCG) improvements
_ Presented in board meetings, as well as involved in other business meetings.
Ask.com Edison, NJ
Research Engineer May 2004 – Feb 2007, Manager Feb 2007 – Mar 2007
Ask.com a division of IAC Search & Media is one of the top search engines in the US.
Originally known for question answering, it is still a leading mobile search and ads company.
_ Invited presenter at the NATO MMDSS conference in Gazzada, Italy, September 2007
_ At Ask.com, defined and created multiple core search, entity-extraction and classification-related technologies used by millions of users every day o Person Name classifier – 3 microseconds
o Automatic Smart Answers based on Wikipedia – covered more than 15% of total traffic
o SQL-based Question Answering engine demonstrated answering more than half of human-generated questions in the area of Baseball
_ Lead a team of engineers to develop and implement internal infrastructure products for improved management of structured data, classification, and data mining
Research Staff Member Nov 2002 – Apr
2004
NEC Laboratories America is a premier research institution which is part of NEC Corporation’s global network of corporate R&D Labs. Known for several high profile research efforts in areas including physics, computer science, semiconductors, and biology.
_ Managed a team of several programmers and students to develop a modular enterprise search technology architecture and demonstration system capable of learning new categories in minutes, and performing category-specific search over a variety of (unstructured) data sources.
_ Developed a prototype enterprise search system (Inquirus 2). This system incorporated many new technologies including: rapid category learning (active learning based), advanced feature selection, SVM-based classification, automated query modifications, intelligent resource routing, multiple-source capabilities, automated query expansion, and search strategies
Education:
_ BSE Electrical Engineering, Magna Cum Laude from University of Michigan, Ann Arbor, MI
4/1994
_ MSE Electrical Engineering, University of Michigan, Ann Arbor, MI 5/1997
_ Ph.D. Computer Science Engineering, University of Michigan, Ann Arbor, MI (my dissertation)
8/2001
Dissertation:
Eric J. Glover, Using Extra-Topical User Preferences to Improve Web-Based Metasearch, Ph.D. Dissertation, University of Michigan, 2001. PDF
Patents:
Please contact for a full list – over 70 issued
Publications:
Halim Abbas Ford Garberson Eric Glover Dennis P Wall, Machine learning approach for early detection of autism by combining questionnaire and home video screening, Journal of the American Medical Informatics Association, Volume 25, Issue 8, 1 August 2018, Pages 1000– 1007, https://doi.org/10.1093/jamia/ocy039
Eric Glover, The "Real World" Web Search Problem, MMDSS NATO Conference, Gazzada, Italy, September 2007. Video Presentation. Please e-mail for an electronic copy of the actual paper.
Eric J. Glover, David M. Pennock, Steve Lawrence, and Robert Krovetz. Inferring hierarchical descriptions, Proceedings of the Eleventh International Conference on Information and Knowledge Management (CIKM'02), November 2002. PS
David M. Pennock, Sandip Debnath, Eric J. Glover, and C. Lee Giles. Modeling information incorporation in markets with application to detecting and explaining events, Proceedings of the 18th Conference on Uncertainty in Artificial Intelligence (UAI-2002), pp. 405-413, August 2002. PS | PDF
Eric J. Glover, Kostas Tsioutsiouliklis, Steve Lawrence, David M. Pennock, and Gary W. Flake.
Using web structure for classifying and describing web pages, Proceedings of the Eleventh International World Wide Web Conference, pp. 562-569, May 2002. PS | PDF | HTML
Gary Flake, Eric Glover, Steve Lawrence, C. Lee Giles Extracting Query Modifications from
Nonlinear SVMs , Proceedings of the Eleventh International World Wide Web Conference, May 2002. HTML PS.Z PS.gz PS PDF BibTeX ©
David M. Pennock, Gary W. Flake, Steve Lawrence, Eric J. Glover, and C. Lee Giles. Winners don't take all: Characterizing the competition for links on the web, Proceedings of the National Academy of Sciences (PNAS), Volume 99, Issue 8, pp. 5207-5211, April 2002. PS | PDF | abstract | HTML | more info
Steve Lawrence, David M. Pennock, Gary William Flake, Robert Krovetz, Frans M. Coetzee, Eric Glover, Finn Årup Nielsen, Andries Kruger, and C. Lee Giles. Persistence of web references in scientific research. Computer, 34(2): 26-31, 2001 PS | PDF
Steve Lawrence, Frans Coetzee, Eric Glover, David Pennock, Gary Flake, Finn Nielsen, Robert
Krovetz, Andries Kruger, and C. Lee Giles. Persistence of Web References in Scientific Research, IEEE Computer, vol 34, no 2, pp 26--31, 2001
Eric J. Glover, Gary W. Flake, Steve Lawrence, William P. Birmingham, Andries Kruger, C. Lee
Giles, David M. Pennock. Improving Category Specific Web Search by Learning Query Modifications, Symposium on Applications and the Internet, SAINT 2001, San Diego, California, January 8--12, 2001.
Frans Coetzee, Eric Glover, Steve Lawrence, and C. Lee Giles. Feature selection in web applications using ROC inflections. In Symposium on Applications and the Internet, SAINT, San Diego, CA, January 8--12 2001.
Andries Kruger, C. Lee Giles, Frans Coetzee, Eric Glover, Gary Flake, Steve Lawrence, and
Cristian Omlin. DEADLINER: Building a new niche search engine. In Ninth International Conference on Information and Knowledge Management, CIKM 2000, Washington, DC, November 6-- 11 2000.
Eric J. Glover, Steve Lawrence, Michael D. Gordon, William P. Birmingham, C. Lee Giles, "Web Search -- Your Way," Accepted to Communications of the ACM
Eric J. Glover, Steve Lawrence, William P. Birmingham, C. Lee Giles, "Architecture of a
Metasearch Engine that Supports User Information Needs," Eighth International Conference on Information and Knowledge Management (CIKM 99), Kansas City, MO, November, 1999
Eric J. Glover, Steve Lawrence, Michael D. Gordon, William P. Birmingham, C. Lee Giles,
"Recommending Web Documents Based on User Preferences," in ACM SIGIR 99 Workshop on Recommender Systems, Berkeley, CA, August, 1999
E. J. Glover, S.R. Lawrence, K.D. Bollacker, C.L. Giles, W.P. Birmingham, G.W. Flake, "A
Metasearch Engine Architecture That Supports Individual Information Needs," NEC Research Institute Technical Report, TR# 99-063, May 13, 1999
E. J. Glover, W. P. Birmingham, and M. D. Gordon, "Improving Web Search Using Utility
Theory," in Proceedings of the First International Workshop on Web Information and Data Management, WIDM 98. Bethesda, Maryland, 1998
Eric J. Glover, Sunju Park, Anil Arora, Daniel Kiskis and Edmund Durfee, "A case study on the evolution of software tools selection and development in a large-scale multi-agent system," in
Workshop on Software Tools for Developing Agents, AAAI 1998. Madison, WI: AAAI
E. J. Glover and W. P. Birmingham, "Using Decision Theory To Order Documents," in Digital Libraries 98, Pittsburgh, PA, 1998: ACM
D. E. Atkins, W. P. Birmingham, E. H. Durfee, E. J. Glover, T. Mullen, E. A. Rundensteiner, E.
Soloway, J. M. Vidal, R. Wallace, and M. P. Wellman, "Toward Inquiry-Based Education Through Interacting Software Agents," IEEE Computer, vol. 29, pp. 69-76, 1996