What Is BERT? – Whiteboard Friday

There’s a lot of hype and misinformation about the new Google algorithm update. What actually is BERT, how does it work, and why does it matter to our work as SEOs? Join our own machine learning and natural language processing expert Britney Muller as she breaks down exactly what BERT is and what it means for the search industry.

Click on the whiteboard image above to open a high-resolution version in a new tab!

Video Transcription

Hey, Moz fans. Welcome to another edition of Whiteboard Friday. Today we are talking about all things BERT and I’m super excited to attempt to really break this down for everyone. I don’t claim to be a BERT expert. I have just done lots and lots of research. I’ve been able to interview some experts in the field and my goal is to try to be a catalyst for this information to be a little bit easier to understand. 

There is a ton of commotion going on right now in the industry about you can’t optimize for BERT. While that is absolutely true, you cannot, you just need to be writing really good content for your users, I still think many of us got into this space because we are curious by nature. If you are curious to learn a little bit more about BERT and be able to explain it a little bit better to clients or have better conversations around the context of BERT, then I hope you enjoy this video. If not, and this isn’t for you, that’s fine too.

Word of caution: Don’t over-hype BERT!

I’m so excited to jump right in. The first thing I do want to mention is I was able to sit down with Allyson Ettinger, who is a Natural Language Processing researcher. She is a professor at the University of Chicago. When I got to speak with her, the main takeaway was that it’s very, very important to not over-hype BERT. There is a lot of commotion going on right now, but it’s still far away from understanding language and context in the same way that we humans can understand it. So I think that’s important to keep in mind that we are not overemphasizing what this model can do, but it’s still really exciting and it’s a pretty monumental moment in NLP and machine learning. Without further ado, let’s jump right in.

Where did BERT come from?

I wanted to give everyone a wider context to where BERT came from and where it’s going. I think a lot of times these announcements are kind of bombs dropped on the industry and it’s essentially a still frame in a series of a movie and we don’t get the full before and after movie bits. We just get this one still frame. So we get this BERT announcement, but let’s go back in time a little bit. 

Natural language processing

Traditionally computers have had an impossible time understanding language. They can store text, we can enter text, but understanding language has always been incredibly difficult for computers. So along comes natural language processing (NLP), the field in which researchers were developing specific models to solve for various types of language understanding. A couple of examples are named entity recognition, classification. We see sentiment, question answering. All of these things have traditionally been sold by individual NLP models and so it looks a little bit like your kitchen. 

If you think about the individual models like utensils that you use in your kitchen, they all have a very specific task that they do very well. But when along came BERT, it was sort of the be-all end-all of kitchen utensils. It was the one kitchen utensil that does ten-plus or eleven natural language processing solutions really, really well after it’s fine tuned. This is a really exciting differentiation in the space. That’s why people got really excited about it, because no longer do they have all these one-off things. They can use BERT to solve for all of this stuff, which makes sense in that Google would incorporate it into their algorithm. Super, super exciting. 

Where is BERT going?

Where is this heading? Where is this going? Allyson had said, 

“I think we’ll be heading on the same trajectory for a while building bigger and better variants of BERT that are stronger in the ways that BERT is strong and probably with the same fundamental limitations.”

There are already tons of different versions of BERT out there and we are going to continue to see more and more of that. It will be interesting to see where this space is heading.

How did BERT get so smart?

How about we take a look at a very oversimplified view of how BERT got so smart? I find this stuff fascinating. It is quite amazing that Google was able to do this. Google took Wikipedia text and a lot of money for computational power TPUs in which they put together in a V3 pod, so huge computer system that can power these models. And they used an unsupervised neural network. What’s interesting about how it learns and how it gets smarter is it takes any arbitrary length of text, which is good because language is quite arbitrary in the way that we speak, in the length of texts, and it transcribes it into a vector.

It will take a length of text and code it into a vector, which is a fixed string of numbers to help sort of translate it to the machine. This happens in a really wild and dimensional space that we can’t even really imagine. But what it does is it puts context and different things within our language in the same areas together. Similar to Word2vec, it uses this trick called masking

So it will take different sentences that it’s training on and it will mask a word. It uses this bi-directional model to look at the words before and after it to predict what the masked word is. It does this over and over and over again until it’s extremely powerful. And then it can further be fine-tuned to do all of these natural language processing tasks. Really, really exciting and a fun time to be in this space.

In a nutshell, BERT is the first deeply bi-directional. All that means is it’s just looking at the words before and after entities and context, unsupervised language representation, pre-trained on Wikipedia. So it’s this really beautiful pre-trained model that can be used in all sorts of ways. 

What are some things BERT cannot do? 

Allyson Ettinger wrote this really great research paper called What BERT Can’t Do. There is a Bitly link that you can use to go directly to that. The most surprising takeaway from her research was this area of negation diagnostics, meaning that BERT isn’t very good at understanding negation

For example, when inputted with a Robin is a… It predicted bird, which is right, that’s great. But when entered a Robin is not a… It also predicted bird. So in cases where BERT hasn’t seen negation examples or context, it will still have a hard time understanding that. There are a ton more really interesting takeaways. I highly suggest you check that out, really good stuff.

How do you optimize for BERT? (You can’t!)

Finally, how do you optimize for BERT? Again, you can’t. The only way to improve your website with this update is to write really great content for your users and fulfill the intent that they are seeking. And so you can’t, but one thing I just have to mention because I honestly cannot get this out of my head, is there is a YouTube video where Jeff Dean, we will link to it, it’s a keynote by Jeff Dean where he speaking about BERT and he goes into natural questions and natural question understanding. The big takeaway for me was this example around, okay, let’s say someone asked the question, can you make and receive calls in airplane mode? The block of text in which Google’s natural language translation layer is trying to understand all this text. It’s a ton of words. It’s kind of very technical, hard to understand.

With these layers, leveraging things like BERT, they were able to just answer no out of all of this very complex, long, confusing language. It’s really, really powerful in our space. Consider things like featured snippets; consider things like just general SERP features. I mean, this can start to have a huge impact in our space. So I think it’s important to sort of have a pulse on where it’s all heading and what’s going on in this field. 

I really hope you enjoyed this version of Whiteboard Friday. Please let me know if you have any questions or comments down below and I look forward to seeing you all again next time. Thanks so much.

Video transcription by Speechpad.com

Digital Marketing News: FTC’s Influencer Guide, Twitter’s New Topic Following, Facebook’s Icebreakers, AR Tops Innovation List, & TikTok’s Easier Sharing

The post Digital Marketing News: FTC’s Influencer Guide, Twitter’s New Topic Following, Facebook’s Icebreakers, AR Tops Innovation List, & TikTok’s Easier Sharing appeared first on Online Marketing Blog – TopRank®.

Source: SEO blog

Digital Marketing News: FTC’s Influencer Guide, Twitter’s New Topic Following, Facebook’s Icebreakers, AR Tops Innovation List, & TikTok’s Easier Sharing

2019 November 8 MarketingCharts Chart

B2B Marketers Are Prioritizing Measurement and Analytics Technologies
83 percent of B2B marketers expect to increase their focus on data analytics in 2020, and 69 percent plan to also make measurement technologies their top investment priority next year, according to newly-released survey data from Dun & Bradstreet and Adweek. MarketingCharts

Twitter Launches New Option to Follow Specific Topics, in Addition to User Accounts
Twitter has begun rolling out the ability to follow some 300 new curated topic lists, in the form of timeline and search prompts, with a full launch slated for November 13, the social media firm recently announced. Social Media Today

LinkedIn Finds That Digital Marketers Are Too Hasty to Try to Determine ROI
Digital marketers rarely take enough time to properly gauge campaign return on investment (ROI), according to newly-released survey data from LinkedIn (client), with 77 percent waiting a month or less for measurement, and only four percent using a period of six months or more. Adweek

Brands on Facebook Messenger get three new features to better connect with customers
Facebook recently announced that its Messenger product has received several new features to help brands connect with customers, with the addition of FAQ icebreakers, Messenger Ad account-selection options, and greater user message reply choices, among other new features. Marketing Land

AR tops list of tech that makes users view a brand as innovative, study says
Augmented reality (AR) tops even artificial intelligence (AI) when it comes to leading indicators of brand innovation, according to newly-released survey results, with 26 percent of consumers listing AR as their most important indicator, followed by AI at 19 percent, facial recognition at 18 percent, and chat-bots with six percent.  Mobile Marketer

TikTok’s new SDK enables video uploads from other apps, services
Social video creation firm TikTok has released its first software-development kit (SDK) — “Share to TikTok” — aimed at making it easier for users to bring in content via third-part apps, as the company looks to drive greater adoption among digital marketers. Marketing Land

2019 November 8 Statistics Image

YouTube Brings Shopping Ads to Its Home Feed, Search Results for the Holiday Season
Google’s YouTube has changed how its home feed features shopping advertisements for the holiday season, with more visible ad exposure within search results, the search giant recently announced. Adweek

The FTC comes out with first user-friendly disclosure guide for influencers
The Federal Trade Commission has released a new influencer partnership disclosure guide, updating its previous version with easier-to-digest messaging on issues such as unpaid partnership disclosures, the independent government agency announced. AdAge

Get faster with the new Speed report in Search Console
Google has given webmasters and marketers alike new tools for measuring a website’s speed, with the launch of a new report in its Search Console utility that breaks down URLs into slow, moderate, or fast ratings, the search giant recently announced. Google Webmaster Central

Twitter Reveals 18 Emerging Trends That Marked U.S. Cultural Shifts Since 2016
Twitter has released a list of 18 cultural trends gathered from an analysis of billions of tweets send since 2016, showing emerging movement to areas of interest to digital marketers including well-being and everyday wonder, the firm recently announced. Adweek


2019 November 8 Marketoonist Comic

A lighthearted look at whack-an-idea by Marketoonist Tom Fishburne — Marketoonist

YouTube Shares Its Top 10 TrueView for Action Direct-Response Ads for 2019 — Adweek

Mark Zuckerberg Announces All Of Facebook’s Future Decisions Will Be Made By The Cube Of Justice — The Onion


  • Lee Odden — Under the Influence(r): How B2B Celebrities Can Boost Your Brand — Brandish Insights
  • Lane R. Ellis — 10 Expert Tips to Help You Grow Your Small Business — Small Business Trends
  • Lee Odden — AMG Newsletter – Why You Need A Marathon Mentality To Succeed — Heidi Cohen
  • Lee Odden — Global Marketing Day in NYC – Meeting A Bunch Of Search Pals

    Barry Schwartz

  • Lee Odden — What We Mean When We Talk About Influencers — Epic Presence
  • Lee Odden — Must-Attend B2B Marketing Conferences for 2020 — Biznology

If you’ve found your own favorite B2B content marketing or digital advertising stories from the past week, please let us know in the comments below.

Thank you for joining us, and we hope you’ll return again next week for more of the latest top digital marketing industry news. In the meantime, you can follow us at @toprank on Twitter for even more timely daily news. Also, don’t miss the full video summary on our TopRank Marketing TV YouTube Channel.

Finding Ideas for a Video Series or Podcast – Whiteboard Friday

Video and podcasts are only growing in popularity, proving to be an engaging way to reach your audience and find ways to talk about your industry or product. But it’s a crowded market out there, and finding a good idea is only half the battle. Join video marketing extraordinaire Phil Nottingham from Wistia as he explores how we can both uncover great ideas for a podcast or video series and follow through on them in this week’s episode of Whiteboard Friday.

Click on the whiteboard image above to open a high resolution version in a new tab!

Video Transcription

Howdy, Moz fans. My name is Phil Nottingham, and welcome to another edition of Whiteboard Friday. Today we’re going to talk about how to come up with a great idea for your video series or podcast. I think a lot of businesses out there understand that there’s just this great opportunity now to do a longer form series, a show in podcast or video form, but really struggle with that moment of finding what kind of idea could take them to the next level and help them stand out.

1. Audience

I think the most common error that businesses make is to start with the worst idea in the world, which is interviewing our customers about how they use our product. I’m sure many of you have accidentally fallen down this trap, where you’ve thought, “Ah, maybe that will be a good idea.” But the thing is even if you’re Ferrari or Christian Louboutin or the most desirable product in the world, it’s never going to be interesting for someone to sit there and just listen to your customers talking about your product.

The problem is that your customers are not a unique group of people, aside from the fact that they use your product. Usually there isn’t anything else that brings them together. For this kind of content, for a video series and podcast to really stand out and to grow in terms of their audience, we need to harness word of mouth. Word of mouth doesn’t grow through the way we often think about audience growth in marketing.

Many of us, particularly in the performance marketing space, are used to thinking about funnels. So we get more and more traffic into the funnel, get more people in there, and ultimately some of them convert. But the way word of mouth works is that a small group of people start communicating to another group of people who start communicating to another group of people. You have these ever-expanding circles of communication that ultimately allow you to grow your audience.

How to find a niche audience

But that means you need to start with a group of people who are talking to one another. Invariably, your customers are not talking to each other as a kind of rule of thumb. So what you need to do is find a group of people, an audience who are talking to each other, and that really means a subculture, a community, or maybe an interest group. So find your group of customers and work out what is a subset of customers, what kind of community, wider culture they’re part of, a group of people who you could actually speak to.

The way you might find this is using things like Reddit. If there’s a subculture, there’s going to be a subreddit. A tool like SparkToro will allow you to discover other topics that your customer base might be interested in. Slack communities can be a great source of this. Blogs, there’s often any sort of topic or a niche audience have a blog. Hashtags as well on social media and perhaps meetup groups as well.

So spend some time finding who this audience is for your show, a real group of people who are communicating with one another and who ultimately are someone who you could speak to in a meaningful way. 

2. Insight

Once you’ve got your audience, you then need to think about the insight. What the insight is, is this gap between desire and outcome. So what you normally find is that when you’re speaking to groups of people, they will have something they want to achieve, but there is a barrier in the way of them doing it.

This might be something to do with tools or hardware/software. It could be just to do with professional experience. It could be to do with emotional problems. It could be anything really. So you need to kind of discover what that might be. The essential way to do that is just through good, old-fashioned talking to people. 

  • Focus groups, 
  • Surveys, 
  • Social media interactions, 
  • Conversations, 
  • Data that you have from search, like using Google Search Console, 
  • Internal site search, 
  • Search volume 

That kind of thing might tell you exactly what sort of topics, what problems people are having that they really try to solve in this interest group.

Solve for the barrier

So what we need to do is find this particular little nugget of wisdom, this gold that’s going to give us the insight that allows us to come up with a really good idea to try and solve this barrier, whatever that might be, that makes a difference between desire and outcome for this audience. Once we’ve got that, you might see a show idea starting to emerge. So let’s take a couple of examples.

A few examples

Let’s assume that we are working for like a DIY supplies company. Maybe we’re doing just sort of piping. We will discover that a subset of our customers are plumbers, and there’s a community there of plumbing professionals. Now what might we find about plumbers? Well, maybe it’s true that all plumbers are kind of really into cars, and one of the challenges they have is making sure that their car or their van is up to the job for their work.

Okay, so we now have an interesting insight there, that there’s something to do with improving cars that we could hook up for plumbers. Or let’s say we are doing a furniture company and we’re creating furniture for people. We might discover that a subset of our audience are actually amateur carpenters who really love wooden furniture. Their desire is to become professional.

But maybe the barrier is they don’t have the skills or the experience or the belief that they could actually do that with their lives and their career. So we see these sort of very personal problems that we can start to emerge an idea for a show that we might have. 

3. Format

So once we’ve got that, we can then take inspiration from existing TV and media. I think the mistake that a lot of us make is thinking about the format that we might be doing with a show in a very broad sense.

Don’t think about the format in a broad sense — get specific

So like we’re doing an interview show. We’re doing a talk show. We’re doing a documentary. We’re doing a talent show. Whatever it might be. But actually, if we think about the great history of TV and radio the last hundred years or so, all these really smart formats have emerged. So within talk show, there’s “Inside the Actors Studio,” a very sort of serious, long, in-depth interview with one person about their practice.

There’s “The Tonight Show Starring Jimmy Fallon,” which has got lots of kind of set pieces and sketches and things that intermingle with the interview. There’s “Ellen,” where multiple people are interviewed in one show. If we think about documentaries, there’s like fly-on-the-wall stuff, just run and gun with a camera, like “Diners, Drive-Ins and Dives.” Carrying on the food thing, there’s “Chef’s Table,” where it’s very planned and meticulously shot and is an exposé of one particular chef.

Or something like “Ugly Delicious,” which is a bit more like a kind of exploratory piece of documentary, where there’s kind of one protagonist going around the world and they piece it together at the end. So you can think about all these different formats and try to find an idea that maybe has been done before in TV in some format and find your way through that. 

A few more examples

So let’s think about our plumber example. Plumbers who love cars, well, we could do “Pimp My Ride for Tradesmen.”

That’s an interesting idea for a talk. Or let’s say we’re going after like amateur carpenters who would love to be professional. We could easily do “American Idol for Lumberjacks or Carpenters.” So we can start to see this idea emerge. Or let’s take a kind of B2B example. Maybe we are a marketing agency, as I’m sure many of you are. If you’re a marketing agency, maybe you know that some of your customers are in startups, and there’s this startup community.

One of the real problems that startups have is getting their product ready for market. So you could kind of think, well, the barrier is getting the product ready for market. We could then do “Queer Eye for Product Teams and Startups,”and we’ll bring in five specialists in different areas to kind of get their product ready and sort of iron out the details and make sure they’re ready to go to market and support marketing.

So you can start to see by having a clear niche audience and an insight into the problems that they’re having, then pulling together a whole list of different show ideas how you can bring together an idea for a potential, interesting TV show, video series, or podcast that could really make your business stand out. But remember that great ideas are kind of 10 a penny, and the really hard thing is finding the right one and making sure that it works for you.

So spend a lot of time coming up with lots and lots of different executions, trying them out, doing kind of little pilots before you work out and commit to the idea that works for you. The most important thing is to keep going and keep trying and teasing out those ideas rather than just settling on the first thing that comes to mind, because usually it’s not going to be the right answer. So I hope that was very useful, and we will see you again on another episode of Whiteboard Friday.

Take care.

Video transcription by Speechpad.com