support tweet-entity expanded_url

[ilf@…]

Starting today, Twitter is forcing their own t.co URL shortener on every post and every account: ​https://dev.twitter.com/blog/next-steps-with-the-tco-link-wrapper

Seizing that opportunity to take action on the general problem that is URL shortening, I'd like BitlBee to support Tweet entities as mentioned in the FAQ: ​https://dev.twitter.com/docs/tco-link-wrapper-faq#How_do_I_render_links_so_that_my_users_don-t_have_to_see_t.co_everywhere

I would like an option for expanded_urlm which shows "the fully resolved media URL": ​https://dev.twitter.com/docs/tweet-entities

[lkraav <leho@…>]

cc

[wilmer]

21:43:31 < roeles> [17] @Wilmer http://t.co/w77cjtb <en.wikipedia.org/wiki/Tarpit_(n…> has some 
                   info that could be handy
...
21:43:31 < AaronToponce> [30] how to write emails to nerds (that they will actually read): 
                         http://t.co/u937yTk <rackerhacker.com/?p=2469>

Does something like that look good?

I'm not committing this yet because it looks like somehow my current version breaks fixing of truncated tweets. :-/

[wilmer]

changeset:devel,809.

[ilf <ilf@…>]

This is not really working for me yet.

Out of 1514 t.co URLs in my timeline from yesterday, I only saw 18 display_url.

Also, I'd really like an option to see expanded_url.

​https://dev.twitter.com/docs/tco-link-wrapper/faq#How_do_I_render_links_so_that_my_users_don-t_have_to_see_t.co_everywhere

[wilmer]

ilf, I see the same in Twitter on Android. Please verify that in your case Twitter really *is* supplying the required info.

[wilmer]

Also, see the 809 commit msg for my reason to ignore expanded_url.

[ilf@…]

Please verify that in your case Twitter really *is* supplying the required info.

Hey Wilmer, how would I verify that? It works in my Twitter for Android. And Twitter Web Client of course.

[ilf@…]

Also, see the 809 commit msg for my reason to ignore expanded_url.

Yeah, it sais:

I'm going for this way of rendering even though full URLs are also available to me because A) it puts me on the safe side wrt Twitter ToS and B) some full URLS may really be very long.

B is what I want :)

About A: I didn't find anything in the ToS. Even if so, an option to enable it would be nice for people like me, you can still leave it off per default.

[wilmer]

I don't think t.co is literally included in the ToS but officially the point of t.co is to block malicious URLs. A client completely bypassing it could possibly be against their policy. It's not that I care that much about their policy, but for things I don't care much about anyway the path of least resistance is often attractive. :-P

So on my Android Twitter client I see many t.co links. I suspect that clients themselves are responsible for including entities in their msgs. Seems ridiculous though.

[ilf@…]

But we all know the real point of t.co is for Twitter to centralize highly used features (like URL shortening and images) in their own infrastructure, so they can control, monitor, analyze and monetize from it.

So fuck 'em, give us the full URLs :)

[wilmer]

Of course we all know that, but I don't care.

Showing full URLs could be seen as circumventing their silly little scheme and could result in BitlBee's API access getting blocked. I have no interest in battles like that since it would require everyone to upgrade their BitlBees.

I'll see what I do.

[ilf]

I still don't really see URLs expanded. Out of 1264 t.co URLs I saw yesterday, only 27 were expanded_urls. And only two of those weren't 4sq.com. I see them all on Twitter for Android. What am I doing wrong?

[Wilmer van der Gaast <wilmer@…>]

In the cases I've investigated, the info was simply missing in the XML from Twitter. Please confirm that this isn't the case for you.

[wilmer]

I've noticed one case with media (as opposed to url) entities that BitlBee was missing. That's fixed in changeset:devel,912.