I wanted to find my most favorited tweets of all time. I figured I should know what those are so I can re-home them (or some version of their spirit) elsewhere in case the ol’ bird kicks the can. Turbulent times over there.
Twitter has some built-in analytics tooling that I couldn’t quite figure out how to make work for “all time”. I couldn’t find any third-party thing that seemed to work for that time frame either.
But you can request and then download an archive of all your Twitter data. I’d highly recommend snagging your archive and tucking it away somewhere, just so you’ll always have that information should you want it someday. Things are so weird over there I wouldn’t bet a nickel that exporting all your data is going to continue to work.
Turns out I’ve tweeted 40,586 times. Took me about 24 hours to get.
With an archive of all my tweets, I figured I could extract the most favorites ones. I am, ostensibly, a computer programmer. Let’s do it, and do it in Go because whatever Go is cool and fast and good.
Step 1: Get the data
Download the archive and you’ll get a .zip you can open:
The juicy file is data/tweets.js.
Step 2: Turn it into JSON
I wanted JSON data, so what I did was remove the window.YTD.tweets.part0 = part and made it valid JSON. Just to avoid touching the gigantic file too much I named the array, so with alternations, my new data/tweets.json file is like:
{
"tweets": [
{
"tweet": {
"favorite_count": "9",
"id_str": "1589326526302113792",
"full_text": "pee pee poo poo"
},
},
{
"tweet": {
...
}
},
...
]
}Code language: JavaScript (javascript)It feels like somewhat of a silly structure. I feel like it should be a top-level array and they each of them don’t need the parent “tweet” naming. But whatever, it’s valid JSON.
Step 3: It’s Go time
I made a main.go file in the root of the export. Ultimately I needed to json.Unmarshal the JSON to use it. But Go needs to have all the data typed, so to match that weird structure:
package main
type Tweets struct {
TweetParents []TweetParent `json:"tweets"`
}
type TweetParent struct {
Tweet Tweet `json:"tweet"`
}
type Tweet struct {
FavoriteCount string `json:"favorite_count"`
ID string `json:"id_str"`
FullText string `json:"full_text"`
}Code language: JavaScript (javascript)There is way more data in that JSON, but all I really wanted was the favorite count, the text, and the ID so I could construct a URL. Fortunately, Go lets you parse the JSON with an incomplete type (which seems unnaturally chill of Go, but hey, cool).
So open the file, snag out the byte data, and unmarshal it:
func main() {
jsonFile, err := os.Open("./data/tweets.json")
if err != nil {
fmt.Println(err)
}
byteValue, err := ioutil.ReadAll(jsonFile)
if err != nil {
fmt.Println(err)
}
err = json.Unmarshal(byteValue, &tweets)
if err != nil {
fmt.Println(err)
}
...Code language: JavaScript (javascript)Error handling for days, Go style.
Now tweets is a big ol’ array of data to munge. 40k is no problem for Go.
Step 4: That Cool Refreshing Slice to Sort
At this point I got annoyed enough at the weird structure I found it prudent to make just a regular []slice of tweets, so I forced the issue:
type RegularTweetsSlice []Tweet
var allTweets RegularTweetsSlice
for i := 0; i < len(tweets.TweetParents); i++ {
allTweets = append(allTweets, tweets.TweetParents[i].Tweet)
}Code language: HTML, XML (xml)Still annoying: the favorite count is a string of an int. 🙄 So when we go to sort it, we gotta coerce into an int first. But hey Go 1.18 apparently made this a lot easier:
sort.Slice(allTweets, func(i, j int) bool {
favCountA, _ := strconv.Atoi(allTweets[i].FavoriteCount)
favCountB, _ := strconv.Atoi(allTweets[j].FavoriteCount)
return favCountA > favCountB
})Code language: JavaScript (javascript)Now allTweets is an in-order slice of every tweet I’ve ever twote.
Step 5: Do something with them.
I have no idea what that is yet, but hey at least I got the data I want.
For now I can just barf them out:
func printOutput(Tweet Tweet) {
fmt.Println("Number of favorites:", Tweet.FavoriteCount)
fmt.Println("Tweet:", Tweet.FullText)
fmt.Println("URL:", fmt.Sprintf(`https://twitter.com/i/web/status/%s`, Tweet.ID))
fmt.Println("----")
}Code language: JavaScript (javascript)At the end of my main function then I can just spit out 10 or so:
for i := 0; i < 10; i++ {
printOutput(allTweets[i])
}Here’s mine top 10:
Number of favorites: 3229
hey ive been a little busy this week ive started a new role in support
Nov 3, 2017 — My daughter Ruby was born
Number of favorites: 3240
Very clever CSS trickery here from Carter Li. If you declare a CSS Custom Property as an <integer>, then it can be interpolated as an integer. Meaning it can be animated!
Oct 8, 2020 — CSS-Tricks article about a cool thing @property can do
Number of favorites: 3319
You know like when your parents ask you a technology question and you don’t know the answer but you figure out and fix the problem anyway through troubleshooting?
That’s why you should apply to basically any job you want.
Dec 22, 2018 — Truth
Number of favorites: 3348
I’d watch a documentary series of developers giving a tour of their codebases.
Jan 5, 2019 — I changed my mind now I only like art theft documentaries
Number of favorites: 3747
I used to be all like
console.log("myVariable: ", myVariable);and now I be all like
console.log({ myVariable });cuz DevTools automatically logs it with the name then.
Nov 10, 2020 — I still forgot to do this a lot.
Number of favorites: 3760
Serverless.
Apr 8, 2018 — This meme I did.
Number of favorites: 4006
who called it a front-end dev and not a diveloper
Oct 11, 2018
Number of favorites: 4584
Write the article you wish you found when you googled something.
Oct 30, 2017 — My advice for writing for CSS-Tricks
Number of favorites: 5633
hey mom that dumb website i started in your basement now has half a million twitter followers.
Nov 4, 2021 — A tweet celebrating a Twitter milestone
Now if I could only find a programmatic way to find what my favorite tweets (of my own) were.
Leave a Reply