vincent - Network

vincent: New cool feature on

New cool feature on bird.makeup this morning: pinned tweets!

They show as native pinned toots and they are not affected by the 100 top tweets limitation.

vincent: To fix the crawling,

To fix the crawling, I'm going to need to start using Twitter accounts to fetch new posts. This is getting complicated for the time I can put on this as a side project, so I'm going to rely on Nitter implementing this, then I'm going to use that. You can track progress here: https://github.com/zedeus/nitter/issues/983

There is another thing I can fix natively in bird.makeup though. I noticed pinned tweets are there in user lookups. I can forward this, and the top 100 limitation doesn't apply to that.

2/2

vincent: I'm getting a lot of

I'm getting a lot of question about what's currently working with bird.makeup, so here it is:

The system is mostly back online, but there are some major limitation when crawling new posts.

- To be forwarded, a post has to be in the top 100 of an account as measured by likes
- There are no retweets being forwarded right now

1/2

vincent: One really useful th

One really useful thing Nitter could do is to log the id of new tweets (and their author) as it encounters them during normal operation. Bird.makeup could simply hook into that byproduct and it can already figure out the rest on its own.

I’ll sleep on it and if I still like it I’ll propose it to the nitter dev

vincent: And no retweets, but

And no retweets, but I did notice I can get pinned tweets though

vincent: And we are back*, ba

And we are back*, baby!

* with the limitation on popular tweets for now

vincent: Currently setting up

Currently setting up a mastodon instance and it looks like all requests have a 20 second delay added. It's driving me crazy!

Anyone knows what's up? My google-fu is giving me nothing

#MastoAdmin

vincent: So they are all gree

So they are all green right now, and the fetching tweets part works from my computer and CI, but not from any of the servers... The approach of fetching tweets from the embeds doesn't seem to work....

FML, and back to the drawing board!

vincent: I will continue with

I will continue with something homegrown for users and individual tweets, but I'm thinking if I should add a dependency on nitter to fetch timelines 🤔

The lead dev has timelines working again already: https://github.com/zedeus/nitter/issues/983#issuecomment-1684070343

While having everything built-in is neat, strategies to crawl are getting more complicated and it's starting to take a lot of time to re-implement things in C#

vincent: More checkmarks are

More checkmarks are green today!

vincent: Looks like Twitter h

Looks like Twitter has made it so that when you look up a profile without being logged in, you see posts ranked by likes instead of by time. This will be a problem…

vincent: Guys I didn’t change

Guys I didn’t change anything yet and bird.makeup started working again 🤯

vincent: Looks like Twitter h

Looks like Twitter has removed the endpoint bird.makeup was using to fetch profiles...

I'll do another reverse engineering round to figure out another way

vincent: Just hooked up bird.

Just hooked up bird.makeup to tracing software for the first time and it's really fascinating!

For exemple, this is an histogram of how fast Twitter answers queries for recent posts by an account.

Look how slow it is! There are a few speedy one at 50~ms, but most are >300ms, which would indicate that they are not cached.

I would have thought it was the other way around, most accounts cached a few that have posted recently that are not cached

vincent: I'm testing stuff re

I'm testing stuff regarding authorized_fetch, but I would need a mastodon server that has it turned on. Know any?

vincent: Some interesting fed

Some interesting fediverse stuff from the BBC! https://www.bbc.co.uk/rd/blog/2023-07-mastodon-distributed-decentralised-fediverse-activitypub

vincent: Also I'm going to ma

Also I'm going to make it deploy automatically from sourcehut when I push new changes, this is going to save me so much time!

vincent: I did it! I now have

I did it! I now have a kubernetes cluster!

Here is the config for bird.makeup, for those curious: https://git.sr.ht/~cloutier/bird.makeup/tree/master/item/k8s/dotmakeup.yaml

I'll see how it behaves in the next week or so, and then I'll start consolidating the various servers I have to it.

Also it's only in web server mode, it doesn't crawl yet. I will use the ordinal of the statefulset as the shard ID to enable crawling. Right now it has to be manually set per node

vincent: Getting lots of erro

Getting lots of errors from infosec.exchange on follow requests on bird.makeup, have they changed something recently?

vincent: Deploying new versio

Deploying new versions of bird.makeup is starting to take a while with so many boxes, I think it's time I move it to a kubernetes cluster