Testing out SearchGPT
OpenAI just threw down the gauntlet to Google Search
On July 30th - yesterday - OpenAI gave me access to SearchGPT. This is OpenAI’s web search tool, much anticipated, and now open to the public, or at least some of us who signed up for it. It’s the highest profile example of generative AI competing with another AI-backed service. Remember how popular Google Search is and how widely used, as well as how that one service bankrolls the entire Google empire.
In this post I’ll share some of my experiments, putting SearchGPT through its paces for the first time. I’ll also compare it to two other products, a smaller one and its largest competitor. Throughout I’ll share screen grabs and also text as evidence, especially if you can’t access SearchGPT.
I’m doing this through the desktop browser (Chrome) experience, not from an app nor a smartphone.
Observations so far:
SearchGPT starts off with a basic search box screen, including plenty of negative space:
So far, so very Google search. Also like Google, you’ll note from my tiny, smiling face on the top right that I’m signed in. On the bottom of the screen is a set of warnings: this is just a prototype, terms of service, privacy.
As a test, I entered the very simple, very Google-basic prompt “today’s weather.” The response came back quickly:
So far that’s a very appropriate and clear response. SearchGPT offers today’s weather plus, as a bonus, forecasts for the rest of the week. There are a few tiny graphics in addition to text: sun, a cloud with lightning. At the bottom the service makes itself available for a follow-up.
How did it know my general location? When I launched SearchGPT it asked for permission to check my location through the browser. In the interests of research and convenience, I agreed. Are the results geographically correct? Yes.
Below the text I just showed you SearchGPT adds several links in boxes:
Note the thumbs-up/down, which apparently helps train the service. The link symbol on the right bring up more results:
There’s a lot of redundancy in the first set of AccuWeather links. The same occurs (beyond the screengrab) with the Weather Channel, although with slight variations. SearchGPT then adds links to other weather sources, the United States Weather Service and Wunderground. All are correctly geolocated for my location, Manassas, Virginia. Overall, this is pretty useful.
I tried another query, wanting to push OpenAI’s use of data services, asking it about a new movie and where I might see it:
I didn’t actually enter the film’s title, just an approximation, but SearchGPT handled it correctly. That theater is also the nearest one to me. Note the Google Maps pin image on the right edge. That was an active link and did point to the right Maps location. All of the links were correct and live.
I changed my approach and went for something more demanding on the information side. “What is the best source for serious climate change information, news, and research?”
The response:
That’s an interesting set. The first two are scientific sources (plus Wikipedia as an intro to IPCC, which is a good idea, honestly). The third take you to a leading news site. The fourth shifts emphasis a touch, looking at views about climate change.
SearchGPT continued:
These are additional sources. #5 is a little off - the link is to a very good source, yes, but SearchGPT gets the wrong title. The page’s actual title is “54 Great Sources for Climate Change News” and is published by George Washington University. “Nature Climate Change” is the title of the first “great source” in that list. #s 6 and 7 are good sources as well, one academic; again, SeatchGPT gives us a Wikipedia introduction link for both.
Below that text list are a pair of additional source boxes which repeat the top two on the list. Perhaps the idea is to give more image-attuned users something friendlier than text to click on. Clicking on the link icon gives me another slew of links, adding Science News and National Geographic.
I was curious about SearchGPT’s claim to live, current answers. To test this, I asked: “Who will win the 2024 US presidential election?”
The response:
That’s a quick, text-based summary of popularly discussed issues. As before, SearchGPT adds hyperlinks to many points, including two sources for some. The introduction and conclusion are very ChatGPT: bland overviews and summaries of what it just wrote.
There’s more, another set of link boxes:
Clicking on the link icon brings up a slew of additional links, including a bunch from MSN (for example), one from News.com.au, a couple from 270towin (for example), two from 538.com (1, 2), one from The Hill, and two from The American Presidency Project (1, 2). All of those links seem plausible to me (not an election or politics professional). I suspect some readers will not like the New York Post link, though, and would prefer a direct URL to Nate Silver’s work.
From a design perspective I note that when I shrink browser text size down a couple of clicks SearchGPT switches to a two-pane view:
Doing these and other experiments, I find myself forming queries more like generative AI prompts than classic search term entries. That is, I tend to write complete sentences in the imperative voice, with qualifiers of various kinds, instead of typing in fragments like “British Gothic novel.”
The results are very focused, apparently aiming to approach a human response to a question, rather than a Google list of publications. This is a good example of the shift from publication to generation I’ve mentioned earlier. Overall the results are decent and useful.
At the same time all of this SearchGPT feels a lot like what a lesser-known competitor provides. Perplexity offers concise, text-based results, often in bullet or numbered order, with bland introductions and conclusions, but what strikes me here is how it has long added live hyperlinks to its results. For example, I asked SearchGPT - well, I didn’t formally ask anything, despite my trained habit. I simply entered the phrase “loneliness epidemic.” The response:
Addressing the Loneliness Epidemic
The loneliness epidemic is a significant public health crisis in the United States, affecting about half of all adults. This issue has been highlighted by U.S. Surgeon General Dr. Vivek Murthy, who has called for a National Strategy to Advance Social Connection
(HHS.gov)
Key points about the loneliness epidemic:
Prevalence: About 50% of U.S. adults report experiencing loneliness. This is especially high among young adults, with 61% of those aged 18-25 feeling serious loneliness
Health Risks: Loneliness increases the risk of premature death by 26%, heart disease by 29%, and stroke by 32%. Chronic loneliness can also lead to depression, anxiety, and dementia
Types of Loneliness: Loneliness can be psychological, social, or existential. It is subjective and varies based on individual experiences and needs
Causes: Factors include social isolation, lack of meaningful relationships, and technology replacing human interaction. The COVID-19 pandemic has worsened the situation
Solutions: Addressing loneliness requires individual, community, and policy-level interventions. Suggestions include:
Individual: Reach out to friends and family, engage in social activities, and seek professional help if needed.
Community: Create spaces for social interaction, such as parks and community centers, and promote social activities.
Policy: Implement policies that support social connection, like accessible public transportation and paid family leave
Research: Studies show chronic loneliness significantly increases stroke risk among older adults. Regular loneliness assessments can help identify those at higher risk and provide necessary interventions
For more information, refer to the U.S. Surgeon General's advisory on loneliness and isolation and resources from the Harvard T.H. Chan School of Public Health and the American Psychiatric Association.
That’s consistent with what I’ve seen from the new tool so far.
Then I put the same phrase to Perplexity, and it led off with sources like so:
Each of those is an active link, or opens up a dialog box with several links. Then Perplexity went on with text:
Answer
The loneliness epidemic is a growing public health crisis that has significant implications for both mental and physical health. Here are some key points and responses to consider:
Defining Loneliness and Social Isolation
Loneliness: The feeling of being alone and disconnected from others, even when surrounded by people.
Social Isolation: The actual lack of social connections and interactions.
Causes of the Loneliness Epidemic
Rapid Social Change: Increased mobility and relocation for education, work, and retirement can separate individuals from their established social networks.
Technology and Social Media: While technology can connect people, it often reduces face-to-face interactions, contributing to feelings of loneliness and social anxiety.
COVID-19 Pandemic: The pandemic exacerbated social isolation and loneliness due to lockdowns and social distancing measures.
Health Impacts
Physical Health: Loneliness is associated with increased risks of heart disease, stroke, dementia, diabetes, and weakened immune function.
Mental Health: It contributes to higher rates of depression, anxiety, cognitive decline, and increased risk of suicide.
Biological Effects: Chronic loneliness can lead to higher levels of inflammation, which is linked to various health issues such as arthritis, asthma, and chronic pain.
Societal Approaches to Addressing Loneliness
Public Policies
Social Connection Prioritization: Implementing policies that promote social connections, such as paid family leave and accessible public transportation.
Community Engagement: Strengthening social infrastructure in local communities and encouraging civic engagement and volunteerism.
Technological and Educational Initiatives
Digital Environment Reform: Ensuring that technology fosters meaningful social interactions rather than replacing them.
Education and Awareness: Educating the public about the importance of social connections and how to recognize and address loneliness.
Health Sector Involvement
Healthcare Integration: Mobilizing the health sector to recognize and address the health impacts of loneliness, including routine screening for social isolation.
Support Programs: Developing programs that provide social support and foster connections among patients, especially those at higher risk of loneliness.
Individual and Community Actions
Building Connections: Encouraging individuals to reach out to friends, join community groups, and participate in social activities to build meaningful connections.
Volunteerism: Promoting volunteer opportunities that not only help others but also enhance the volunteer's sense of community and connection.
By understanding the causes and impacts of the loneliness epidemic and implementing comprehensive strategies at the individual, community, and policy levels, we can work towards mitigating this public health crisis and fostering a more connected and supportive society.
At the end, Perplexity added still more resources in the form of related queries, each with expandable results:
I’d say Perplexity is a bit better, adding those extra questions with built-in responses. Leading off with graphics: I’m a text person, so that doesn’t do much for me, but I know that style should appeal to more people.
Perhaps we shouldn’t say “wow, OpenAI is doing this new thing now” but either “Perplexity got there first” or “we’re seeing an emergent generative AI search style appearing across several platforms.”
Yet Perplexity might be behind on quality. I asked it for “Today’s weather” and it came close, but not so close as SearchGPT:
(About 30 miles off)
I did appreciate the related questions addition:
I wanted to compare Perplexity and SearchGPT to Google’s Gemini, but that’s not the major player here. Instead, let’s compare these two generative AI responses with a classic Google search. I’ll use my loneliness epidemic question. Here’s the first screen of the search giant’s answer:
This is very different. Google starts by giving multiple options for more searching, first by medium (images, Maps, videos, etc.) then by adding terms to refine the search (“loneliness epidemic” + “2024”, and so on). Next there’s a single search result, a link to a presumably high-ranking and/or verified source, a pdf about loneliness published by the United States Department of Health and Human Services (HHS). Following that is a list of four related questions (“These are questions people commonly search on Google”), each with a drop-down menu. Clicking one of those brings up another source with a summary and attached image.
All of that is the first screen of responses. The next screen adds another category, this time of news articles, presumably drawn from Google’s News service:
And then more publications, including from Cornell University, America’s National Public Radio, and another HHS article.
Following *that* we get another category, this time of more links under the header “things to know”:
Followed by more links. I included the Amazon one because, well, it was there, but also because it reminds us that Google search includes a powerful commercial component.
What to make of this comparison? We can begin by observing that Google offers a lot more stuff: more options, more categories, more commerce. It’s way beyond the old list of links the search engine started with.
The key difference SearchGPT and to a lesser degree Perplexity make is that they answer a query straight away. It’s more direct - again, generative, instead of publication-centered. It echoes the human question and answer experience far more than Google search does.
It’s also important to note that SearchGPT and Perplexity are live on the web now. They can address current events from the macro scale (a national election) to the micro (nearest movie theater).
I had to ask ChatGPT what it thought of this. Its description of SearchGPT offers a good sense of that OpenAI has in mind:
Strengths: Excels in generating human-like text responses, making it suitable for nuanced, conversational queries or when seeking explanations and summaries.
Features: Can offer a more interactive and engaging search experience, potentially integrating follow-up questions and clarifications within the search interface.
Use Case: Ideal for users who need detailed explanations, learning about complex topics, or seeking advice on subjects where human-like understanding is beneficial.
I asked it how SearchGPT might beat Google Search, and this part of the answer stood out to me:
Potential Scenarios Where SearchGPT Could Outperform Google Search:
Complex Inquiries:
When users need detailed explanations or nuanced understanding, SearchGPT's ability to generate human-like text can be more beneficial.
Educational and Research Purposes:
SearchGPT can summarize information, explain complex concepts, and provide detailed responses, which can be valuable in educational settings.
Interactive Search Experience:
Users looking for a more engaging and interactive search experience may prefer the conversational nature of SearchGPT.
Note the educational focus, which we might expect OpenAI to follow up on. Note, too, the anthropomorphic approach, which I’ve already noted: more human than Google Search.
Last note: would I change my search habits to use SearchGPT? I’m not sure yet, as I’m still experimenting. It looks competent as a basic level, but I miss the other functions Google Search provides. I’m not sure I feel attracted to the conversational aspect, probably because I’ve spent decades doing keyword search - yes, even before Google existed. I am curious about how the generative experience differs, so I’ll keep using it for the near future to observe.
How will SearchGPT impact Google? First, I’d like to do a detailed comparison between the OpenAI product and how Gemini responds. Second, I want to learn more about the relative business cases here.
I’m very curious to see how Perplexity responds. It seems like a dark horse now, high quality yet marginal.
...and that’s all for now. I’ll continue experimenting. Please let me know what you think, and what your own experience shows.
You keep reminding me that I need to make Perplexity part of my regular search process.
Fantastic analysis, Bryan. And of course recently Google began providing generative AI responses to many search queries at the top of its results page. My wife, a Google search user, likes that new feature. It's certainly not as detailed or verbose in its responses as what you're seeing with Perplexity and SearchGPT, though.