Modify

#1202 closed defect (fixed)

Twitter: tweets with embedded tweets sometimes(?) not shown

Reported by: levu@… Owned by:
Priority: normal Milestone:
Component: Twitter Version: devel
Keywords: Cc:
IRC client+version: Client-independent Operating System: Linux
OS version/distro: Ubuntu 12.04

Description

Sometimes tweets with embedded tweets (= link to a tweet inside) don't get shown. I experienced this now a hand full of times over the last days, starting when twitter announced their "retweet with comment" feature, which essentially is just tweeting with a link to some other tweet inside.

Some other tweets with links to a tweet inside were displayed indeed.

I looked into the twitter API, a tweet object looks the same it looked some weeks ago, so I guess there were no changes.

Since there were no missing bitlbee-tweet-IDs (those 2digit-hex-numbers) in the time range the tweet was posted, this is not some client filtering or so.

Version is BitlBee 3.4-0-g61d21e5-release-3.4-git API version 030400

Attachments (0)

Change History (14)

comment:1 Changed at 2015-04-09T19:40:26Z by dx

It could be some client filtering. Maybe it's considering them as retweets (with no extra comment) and deduplicating them as such, because they were already retweeted / quoted before.

comment:2 Changed at 2015-04-15T22:31:07Z by wilmer

Yes, would be interesting to see whether that's explained by dx' arguments. Though the format appears to be different. An example I've just grabbed from my stream:

http://p.gaa.st/view/51702364

Nested message is a "quoted_status", not a "retweeted_status". BitlBee doesn't handle that at all. (Also, note the lack of useful results for https://www.google.co.uk/search?q=twitter+api+"quoted_status" .. nice documentation guys! :-/)

I wonder whether these can be nested, I'll assume not. But could it be native-retweeted? It *looks* like it can't be: I've just clicked on a message like that, and it shows me the dialog where I can add a comment, but the quoted Tweet is shown as just the added text and a twitter.com/... URL at the end.

comment:3 Changed at 2015-04-16T18:57:51Z by dx

I really have no idea. Given that pastebin, my theory is wrong, and everything should work.

The developer forums don't mention this at all, the REST api doesn't return quoted_status for the same tweet wilmer linked.

Maybe a server side issue while they roll out this feature?

I'd appreciate any reports that this is still happening, and "me too" comments in general, if anyone else is having this issue.

comment:4 Changed at 2015-04-16T23:34:14Z by dx

I poked two random twitter users who appeared in my tracking stream complaining about twitter issues, and i thought they referred to this one, but nope (both were about old bugs which are fixed already)

Sooo as far as I can see, no one else reported this bug (To be fair, it's hard to notice it)

But yeah, I'm giving this a few weeks, if it doesn't happen anymore, this could be closed blaming the server.

comment:5 Changed at 2015-04-22T09:11:13Z by wilmer

Either that or repurpose this bug for decoding these Tweets properly. I'm already tired of clicking on the twitter.com links because seeing the added comment only is super-unhelpful. :-(

comment:6 Changed at 2015-05-02T22:21:33Z by wilmer

Parsing this now, this is going to be the format:

00:19:03 < BitlBeeTest2> [13] ok nog eentje <https://t.co/eQaBK01Xvk> [@Wilmer: After some hours on 
                         the UK roads,I don't see why people fear self-driving cars. Can't be hard 
                         to have computers drive at least that "well".]

Funny discovery: It appears that not all feeds include a "quoted_status" property yet. I'm getting it from the streaming API only for now. It's not documented so who knows what they'll do with it/for how long this is going to work..

comment:7 Changed at 2015-05-03T02:25:40Z by dx

Sorry, ticket takeover cancelled!

While reviewing wilmer's new stuff I noticed that I was also missing *some* quoted tweets, the original issue here. I posted this thread in the developer forums:

https://twittercommunity.com/t/some-retweets-with-comments-are-missing-filtered-out-from-user-streams/36826

I'm still mostly blaming the servers, but who knows. Hopefully some twitter employee will give it a look.

comment:8 Changed at 2015-05-03T08:43:39Z by wilmer

Sorry, ticket takeover cancelled!

*shakes fist*

It indeed seems most likely that they're not appearing in the streaming API sometimes. As I said above there's some oddity with them already anyway. Lame. :-(

I can restore my hack to dump every message on the stream to confirm this if necessary. Or just temporarily disable stream deduping.

comment:9 Changed at 2015-05-06T10:06:25Z by wilmer

So Twicca beat me to it, they did a release yesterday with quoted_status support. And not just in the streaming API...

No source code available to easily see what's their trick.

comment:10 Changed at 2015-05-10T07:30:27Z by dx

Cc: meh@… added

No source code available to easily see what's their trick

I had a look. It's not quoted_status, it's just a REST api request to /1.1/statuses/show/<id>.json right after the request to home_timeline. Meh.

comment:11 Changed at 2015-05-10T07:30:54Z by dx

Cc: meh@… removed

whoops, sorry for that

comment:12 Changed at 2015-05-10T08:18:57Z by dx

More relevant to this ticket: added a post to that thread, in which I believe I've found what causes this. It applies only to quoted statuses that start with a @mention, and as far as I can see it's definitely twitter's fault

@OP: can you confirm that the tweets that disappeared were quoting other tweets that started with @?

Because apparently all the examples I have of missing tweets are my own, no one else reported it. But it's reproducible, so it's gotta be this.

comment:13 Changed at 2015-05-10T14:24:26Z by levu@…

I'll have a look into it, couldn't reproduce it on the first try, but I'll keep you informed.

comment:14 Changed at 2015-05-27T01:51:22Z by dx

Resolution: fixed
Status: newclosed

Quoting myself from the twitter thread, 15 days ago:

Did some more tests, now with just curl + signatures for the userstream generated with OAuth Tool.

And, uh, either it's fixed, or my theory that it's related to the fact that they started with mentions is wrong. Now I kinda regret not testing more than two times. I repeated it 12 times, all of them showed up, both in the curl stream and my client

I think I'm just going to keep looking for more of those like i did before (comparing stream results with twitter web contents) for a few more days until I get bored of wasting time with this.

And I haven't seen this happen again.

Closing this as "fixed" because wilmer did something, and I like to close stuff as "fixed" when something was done.

Modify Ticket

Action
as closed The ticket will remain with no owner.
The resolution will be deleted.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.