We compete with GitHub. Bing does not show our website

  • This is pure clickbait and really doesn't leave a good impression of this team. Writing a whole article implying they're not being indexed because they are a Github Copilot competitor when they, themselves, don't believe it is farcical.

    Title:

      We compete with GitHub. Bing does not show our website
    
    FTA:

      Now, is it really because we compete with Github? Honestly, probably not, but controversy drums up interest and we need interest so that someone out there on the Internet can tell us what we're doing wrong. [...] we only care about getting as many software engineers to experience the power of generative AI for software development via our free product.

  • Your website isn't quick to find by Duckduckgo or Qwant either. Yahoo doesn't seem to show much either. Startpage.com does find it, though.

    I do consistently get your VS Code addon in the first page of search results, though.

    When I open up your website I don't really see much. A brand name and some download links are all I can find. I'm not sure if there's much for search engines to find, really.

    Your homepage also massively lags my browser for some reason, that can't help in terms of search rankings. It's also continuously downloading megabytes of data for some reason. You should try out your website on a cheap Android phone, your target audience may generally not use those but search engines definitely optimize for them.

    > Now, is it really because we compete with Github? Honestly, probably not, but controversy drums up interest and we need interest so that someone out there on the Internet can tell us what we're doing wrong.

    Mission succeeded, I suppose. You've got another popular link to your website through HN and you're getting free SEO advice on top. Sadly, you won't be able to get Bing support through here, HN tech support posts mostly attract Google and Stripe employees.

  • There's loads of stuff that Bing doesn't index very well.

    If you turn of JS there's almost no actual text on the site; only the FAQ has any serious amount of text. This probably doesn't help. Having the page open also really slows down my laptop and constantly uses ~50%-75% CPU and ~130M memory. No idea if that's a factor, but it takes quite a lot of computing resources to actually make the homepage render any text and perhaps Bing doesn't deal well with that (and even then, it's not all that much text).

    Admittedly I have a somewhat slow laptop, but I can't recall the last time merely opening a webpage slowed down my machine this much, and this includes webapps, so it's really an outlier.

    Either way, Microsoft specifically blocking it is the least likely explanation by far.

  • > Now, is it really because we compete with Github? Honestly, probably not, but controversy drums up interest and we need interest so that someone out there on the Internet can tell us what we're doing wrong.

    It's pretty slimy to imply in the clickbait title that you're being blocked due to competing with MS, when you know well that it's not the reason. Admitting it's clickbait in the last paragraph doesn't make it any better.

    The HN title should really be edited to remove the "We compete with GitHub" part.

  • Last year, Bing and Edge erroneously flagged our website https://sheetjs.com/ as "dangerous": https://i.imgur.com/BvA3zrk.png

    At the time, there was no "Safety Report" to indicate why Bing thought it was dangerous. The report page linked to https://www.bing.com/toolbox/bing-site-safety?url=https%3a%2... and it said "That web page doesn't exist"

    To fix it, we had to register with "Bing Webmaster Tools" (https://www.bing.com/webmasters/about) and raise a support ticket.

    Within a few days, the issue "resolved itself". It's possible that raising a ticket forced some automatic refresh of the indexed data for the domain.

  • For all the hate Google gets, it's crawlers are like 10x smarter than any other search engine. Wouldn't surprise me at all if there was some type of crawling issue that Google figured out and Bing didn't.

    This is a case where I'm not ready to attribute to malice what is very likely incompetence.

  • "The inspected URL is known to Bing, but has some issues with indexation"

    1) Does "indexation" mean what whoever wrote this thinks it means? :P

    2) Why not document what the issues actually are on that page, so that the website owner doesn't have to guess?

  • Someone else recently notice their website wasn't indexed by Bing and they've got a similar message. The whole story is thoroughly documented. Maybe Codeium can follow their steps to resolve the issue?

    https://daverupert.com/2023/02/solved-the-case-of-the-bing-b...

  • If you experience negative seo attacks against your site then being de-indexed by Bing/DDG at some point is to be expected. They are simply not as smart as Google in this respect.

    You can go through the procedure of registering with their webmaster tools equivalent, submit a ticket and wait.

    Or you can simply just wait.

    In my limited experience (6 sites) the result was the same. Eventually it will resolve itself and you'll be back in the index, albeit with some very weird links showing up (an effect of the original attack). This does, at least, alert you to some vulnerabilities you may have previously been unaware of.

  • Start with a robots.txt?

    https://www.codeium.com/robots.txt

    Update: I also see that you serve your site off GitHub... so that might be part of it as well? Bing might not index those sites?

  • The funny thing is, if you search "copilot" on Bing, on the right you'll see "A second pilot of an aircraft". Google returns Github Copilot

  • honestly if you look at the incident history for github, coupled with the ham-fisted tantrum Bing throws if you ask for Chrome ("There's no need to download a new web browser.") , it feels pretty on-point to say Redmond is throwing knuckle sandwiches to stay in the running for any relevance on the internet these days

    https://www.githubstatus.com/history

  • It's easy to underestimate just how difficult it is to get your context indexed and ranked well. Personally I was under the delusion that if you make a solid website, you'll get some organic visitors through search engines from people searching related terms. Turns out that might well be wrong. In the first few months of launching my fairly serious effort website I received a total of 3 organic visitors. While proper SEO and link spamming would surely get me some more, I found it disappointing enough I just gave up

  • > Instead, Codium, an algae genus, gets the sidebar slot while Codiaeum Variegatum, an (admittedly pretty) houseplant takes the fourth spot.

    That makes sense. I never heard of "Codeium" before. Perhaps "Codeium programing" can give a better result for a programing tool. https://www.bing.com/search?q=Codeium+programing

    [Edit: fixed name and bing link]

  • I had an agency client in 2021 with an identical issue. In that case, the agency was able to contact a support representative for Bing's Webmaster Tools that provided a little more info, and some further investigation on my end revealed that the Bing crawler was doing something idiotic that was triggering http error response codes from the application, and the crawler took that to mean that the site was broken or down.

    It was an intersection of multiple stupid things, not maliciousness.

  • Bing's weird with some websites. For instance it will refuse to show the bun homepage (bun.sh) with anything short of a direct copy of the title: "Bun — fast all-in-one JavaScript runtime", even then it gives you a link with a `?ref=hackernoon.com` query parameter for some reason?

    It is happy to show its github repo and a bunch of blogspam about it though.

    It is even happy to link you to bun.sh/install, which immediately downloads a shell script to your computer upon clicking. Bizarre.

  • Bing is known for removing websites for no apparent reason. My website was de-listed for 2-3 months one year ago (personal blog, writing for 10 years, no spam, no ads). It happened recently to another dev blogger: https://daverupert.com/2023/02/solved-the-case-of-the-bing-b...

  • At risk of encouraging the trend of HN turning into tech support, or worse ( guerilla-marketing bait), you should try Bing Webmaster Tools: https://www.bing.com/webmasters/about . If it's anything like Google's Search Console, there's plenty of information and tools to diagnose indexing issues.

  • Why should this codeium suddenly rocket past the already established codeium product that was indexed? Complaining is easy. SEO takes work.

    When searching bing with: codeium coding

    That search will bring this product up in the first position. It's a shame that the specific "indexation" issues aren't shown, but did the author read the linked bing page to see the list of possible issues?

  • It also doesn't show google scholar and twitter for me when I recently switched to try out bing instead of google. With google the scholar and twitter results were in top 3 but couldn't find them with bing, even 4 or 5 pages down the line, while github and pytorch forum posts were available on the first page itself.

  • 1) there is no robots.txt 2) JSON+LD markup is minimal and there is none on the home page 3) your navigation is not fully crawlable - About and Pricing break

  • Maybe try a 5 minute conversation with someone who does SEO for a living before writing up a conspiracy blog. You have essentially no backlinks to your brand new site. Popping you into ahrefs, it looks like you guys weren't even on the map until December. If I filter out the spam domains and isolate just domains that are at least DR 20 with any traffic at all and exclude subdomains, you have links from 16 domains and all but one are automated junk and non-editorial.

    Additionally you have no robots.txt file, and your homepage has no self referencing canonical. TL;DR you don't show up in the index because you are not yet notable. Do some digital marketing.

  • > Now, is it really because we compete with Github? Honestly, probably not

    There's no way of really knowing. Bing is a black box with no transparency, like most search engines.

    One thing though: `Codeium` seems like a generic and vague word. Also I won't remember how to spell it because of the weird `e` before the `i`. I can see why Bing has trouble even recognizing the word. It's optimizing for the correct spelling. Try a rebrand, something catchy, and not something that has a hundred other words that sound like it.

  • I'll throw in my critique, write a <title> or at least read up on the SEO importance of doing so

  • Not sure if changed something, if a MSFT employee saw... but Codeium is the top #1 on my bing search.

  • Bing doesn't show my website either and my product doesn't compete with M$.

  • 4chan also does not show on Bing for me, but it does for some people it seems.

  • Sounds like codeium need some of that copium.

  • Their robots.txt page gives a 404 error.

  • Curious how Codeium compares to Tabnine?

  • Looks like this post is flagged too... I'm also competing with you/co pilot with https://text-generator.io but I'm on Bing, take a deep look into your site/SEO etc and tip of the day for generative companies like ours is that we can generate a whole bunch of examples for SEO.

    Normally HN and Reddit are less moderated than other sites that do a lot of shadow banning, condolences for getting stomped on by large co's and also welcome to the internet

  • "BingGPT, let me access your competitor's website!"

    "I'm sorry Dave, I'm afraid I just don't know what you're talking about."

  • Bing does censor things Microsoft doesn't like, even if they are legal, aren't explicit, or the like.

    Another example I recently ran into: Windows Ameliorated (a set of scripts for heavily trimming Windows 10), famously featured on Linus Tech Tips. Search Google for it, you get the website link as first result. Search Bing... you'll never find the website for it. You will find the archive.org ISO Download link, but https://ameliorated.info will never be returned.