There's a new "Google Jail" for independent wikis

The article says this isn't an issue for subdomains of established domains, but the issues it describes match what we experienced setting up a game wiki for Cyber Knights: Flashpoint at https://cyberknightswiki.tresebrothers.com/ -- almost a year of not having any pages show up in search results, even though we were getting good links to it and there were no better alternatives that should have ranked (we never started the wiki for this game on any centralized site; at most all they had for it was a placeholder page).

Only after months of making sure our Steam news posts were consistently linking to pages within our wiki did we eventually see Google suddenly open the floodgates and start connecting searchers with the info they were searching for.

I've built out highly-ranked content sites before, and none of what I'd learned from getting those started seemed to work with this wiki, for no apparent reason. Google Search Console showed plenty of pages on the wiki were indexed; it just wasn't displaying them. Seeing this post I fully believe the premise that there's simply an arbitrary Google Jail for independent wikis. That sucks for search quality.

When I still payed Dota, there was a wiki hosted on fandom.

The experience was terrible. It wasn't always update-to-date with the latest patch, and on mobile there were so many ads rendering the whole thing unreadable.

The catch is that there was a better wiki, called Liquipedia. But Google never showed it first and I kept getting tricked to click that shitty fandom site.

(It was years ago. But I just tested it a bit and for most Dota-related keywords, fandom is still ranked higher than liquipedia. I don't know if the quality of fandom site has improved though.)

Some of this might just be problems with the websites themselves. With the Path Of Exile 2 Wiki example, I'm looking at the sitemaps. Once you get through the first sitemap of nested sitemaps (ugh, not a fan of those), the very first actual sitemap returns:

  XML Parsing Error: no root element found
  Location: https://www.poe2wiki.net/sitemap/sitemap-poe_wiki-poe2_wiki--NS_0-0.xml
  Line Number 18040, Column 1

(EDIT: Of course, after I post this, it's now working again...)

On to the Hytale Wiki example. I don't know if what I'm about to say applies to Google, but I'm approaching it from the perspective of my tiny dumb indieweb indexer for my personal search engine. It is much easier for me to index from a sitemap rather than try to crawl a website, so I basically look exclusively at sitemaps.

Looking at Hytale Wiki, my process:

This is the point where my tiny dumb indexer would stop. Gzipped sitemaps are part of the standard, but they're relatively rare on the web for small sites. They typically only get used if a sitemap file exceeds the 50,000 pages-per-sitemap limit. In this case, 26 50,000 makes the HyTale Wiki look like a 1.3 Million page site. Do I really* want to index 1.3 Million pages, an estimated 13GB of indexed text data, about a video game I'll probably never play?

My search index is storage constrained, and my indexer is very time constrained. The time I spend indexing your site is time not spent indexing another, possibly higher quality website. So at this point, I'd just grab the front page and disappear... like Google apparently does too.

Of course, the HyTale site isn't 1.3 Million pages, it's only 4,277 articles. That would all fit in the root sitemap file, and that might be the better approach for getting indexed.

I enjoy the this extension that redirects search results from Fandom and other such terrible hosts to their higher quality indie alternatives.

https://github.com/IndieWikiBuddy/indie-wiki-buddy

Subdomains of existing domains are totally fine

Damn, another push to centralizing and walling off user content in 'fandoms', 'reddits' and other closed communities?

Have the internet cards been handed out now, and are we in the end-game?

Will old domains resell for premiums like low-background-radiation steel?

For a while, Fandom was having annoying ads and content that would get past my ad blocker. But a recent update has caused them to block users with ad blockers.

I've been looking for explicit alternatives to Fandom for some content and it's annoyingly sparse.

Just the other day I saw some meme about how if we saw as many ads taking over sites like we do now, that was a sure sign of having your machine getting infected with a virus.

Generally I've found a good portion of websites that block users with ad blockers tend to be the worst offenders with aggressive ads...

I host my blog etc. on Mediawiki. It has a bunch of defaults that make it not SEO friendly. I was absent from all search engines for a year and told a friend[0] how it was funny that I guess Google no longer ranks personal sites and he told me he’d look and see.

It turned out that default Mediawiki stuff doesn’t do a lot of standard SEO stuff. I just did all of it. From the jsonld tags, to the meta description, to making sure the sitemaps were correct. As soon as I did, my site started showing up on Google.

I cannot speak to these wikis’ problems but the first port of call for me would be checking that everything “standard” for this is now done. It’s just a reality of the web these days. I do get random traffic from Google now. Much less than I used to under the old regime in the 2000s but it feels more like a secular change than the flat zero listings I previously did.

I wish I had kept track of everything so that one day I could say what I did but I just followed literally everything, including removing the index.php thing that Mediawiki uses by default. So I cannot even say which actions worked and which didn’t. I can’t even recall which changes to base I did do. Link rel canonical. A robots.txt that disables access to pages that accidentally duplicate content (e.g. permalink pages). Hard to tell. But the difference was stark 0 to hundreds of pages. Literally zero dude. And it happened weeks after the changes and very suddenly.

0: It’s not what he does as a job but he might remember https://www.jrhizor.dev/

I'm going to agree.

I launched a website that is niche a few weeks ago, but totally unique. After a couple of weeks search console wakes up to tell me that they have indexed one page in 80. And that the top search term that is finding my site says exactly the name of one of my pages, yet Google is showing my homepage in search!

Way to help your users

Anyone remember when "Fandom" was called Wikia? It really started going down the gutter when it started TikTokifying itself by trying to act as an entertainment hub. It was supposed to be a neutral resource for people to build community-driven knowledge bases but it turned into some real gamified garbage. I've never really been comfortable with the concept of addressing a "fandom" as one whole anyway, even in general, so the new act is just complete crap to me.

It's not just for new wikis. It's called the Google sandbox in the SEO world and it's something that had been around for years and affects every new domain.

New domains need to prove they can be trusted to get out of the sandbox and external links from trusted sources are the best way to do that and shorten the time spent in the sandbox.

Big wikis like this go online with thousands of pages on a brand new domain. Without any external trust signal it just looks like spam for Google