Threaded Twitter Stream + Search Tox

Name says it all. There’s lots of stuff going on inside this tox, best bet is to read the GitHub readme.

Features:

  • Drag and drop us with custom parameters (still have to enter your own credentials though)
  • Threaded using queues as to not crash your process
  • Logging to keep records of all the Tweets (check the Terms of service for this though)
  • Options for simultaneous real-time streaming API connection and search API
  • Options for using AND based or OR based keyword searches
  • Email ability from inside of TouchDesigner incase of critical error (you’ll have to add your own calls using the pre-built module)
  • Etc etc etc etc

github.com/nVoid/nVoid-Twitter-TouchDesigner

Enjoy! More to come soon.

Can’t wait to give this a crack! thanks heaps.

Just brilliant,thanks again. Much better than what I had done.
Have you guys figured out a way to get the avatars bigger than 48*48 pixels?

hello elburz

thanks for the amazing module, i have noticed a small error when building a show which involves a twitter stream which could go from 1 tweet per hour to 500 per hour.

so when i start the stream and tweet using the correct stream term, a tweet will be received around 2-5 mins later, so this makes it a bit hard when we have a small group or one person trying to see there tweet as quick as possible.

i found if i tweet from the same account or another account straight after the first tweet i can push the first tweet up almost instantly.

im just a bit confused about how this occurs and would be grateful for any help possible

Best Regards
Blake Johnson

The Electric Canvas
Sydney, Aus

Hmm, whats the best method to reproduce the issue?

Are you using a very unique hashtag?

Our dev Matt pushed up an update. There’s a chunk size button. You can read more in the github readme but basically things were getting stuck in the buffer. Should be fixed now if you turn on the smaller chunk size toggle, or leave it on if you’re dealing with larger data sets.

Thanks so much for your help.

I’ll be sure be sure to post up the end result when the build is finished.
We will be using this module on a show at vivid 2016 in Sydney.
So I can’t thank you enough for the awesome module

Best regards
Blake Johnson

Electric canvas
Sydney, Aus

hi, many thx for making this available. i’ve been trying to integrate the ‘media_url’ into the script to get hold of images in tweets. i thought it would be as easy as adding the parameters to the list in the script like so:


me is this DAT.

frame is the current frame.

state is true if the timeline is paused.

Make sure the corresponding toggle is enabled in the Execute DAT.

def start():
return

def create():
return

def exit():
return

def frameStart(frame):
try:
# check queue for tweet
inQ = me.parent().fetch(‘streamInQ’)
value = inQ.get_nowait()

	# prep json
	import json
	json_obj = json.loads(value)
	
	# extract information from json
	tweetText = json_obj['text']
	tweetLanguage = json_obj['lang']
	tweetTimestamp = json_obj['timestamp_ms']
	tweetLocation = json_obj['coordinates']
	tweetHashtags = []
	for i in json_obj['entities']['hashtags']:
		tweetHashtags.append(i['text'])
	userFullName = json_obj['user']['name']
	userScreenName = json_obj['user']['screen_name']
	userLocation = json_obj['user']['location']
	userLanguage = json_obj['user']['lang']
	userProfileImage = json_obj['user']['profile_image_url']
	[b]mediaPostImage = json_obj['media']['media_url'][/b]

	#check tweet for double quotes, and replace with single
	if '"' in tweetText :
		tweetText = tweetText.replace('"', "'")

	# write to table
	op('tweetTable').appendRow([tweetText, tweetLanguage, tweetTimestamp, tweetLocation, tweetHashtags, userFullName, userScreenName, userLocation, userLanguage, userProfileImage, [b]mediaPostImage[/b]])

	# write to console
	mod.writeTo.console('New Tweet in stream from @' + userScreenName)

	#make tweet dict
	twitterDict = {}
	twitterDict["tweetText"] = tweetText
	twitterDict["tweetLanguage"] = tweetLanguage
	twitterDict["tweetTimeStamp"] = tweetTimestamp
	twitterDict["tweetLocation"] = tweetLocation
	twitterDict["tweetHashtags"] = tweetHashtags
	twitterDict["userFullName"] = userFullName
	twitterDict["userScreenName"] = userScreenName
	twitterDict["userLocation"] = userLocation
	twitterDict["userLanguage"] = userLanguage
	twitterDict["userProfileImage"] = userProfileImage
	[b]twitterDict["mediaPostImage"] = mediaPostImage[/b]

	#write to log
	mod.writeTo.log("DEBUG", twitterDict)

except Exception as e:
	# nothing new yet
	pass

return

def frameEnd(frame):
return

def playState(state):
return


but this doesn’t work as i thought it would. is it possible to do this with your component ?

I am also getting an error when i save from time to time but not always:

You can ignore that pickling error, it wouldn’t stop any functionality, just throws the error when saving. Your project should still save regardless. Are you not seeing the media url in the table?

i’m not sure i’m changing things in the right places:

inside the /project1/nVoid_Twitter_Tools1/python

i have opened the

checkForStream execute DAT in textport and added the lines mentioned above in green, then closed the texport tab and saved the project.

i’ve also tried adding a mediaPostImage column to the tweetTable DAT

when i do that i don’t get an error but the tables are not receiving tweets anymore. i remove the added lines from the cript and media column from the tabes and it’s working again.

when i save and close the project and open it up again the script and table have reverted to their initial state and my changes are gone. The project is saving since i see the version count increase and other changes i make to the project stick.

i have also created a table with my authentication credential keys and linked the corresponding cells into the authentication parameters of the tox. this also doesn’t stick, when i save and open again the auth parameters are empty.

i’ve seen in the readme that only these tweet attributes are supported:

“Tweet Message”
“Tweet Language”
“Tweet TimeStamp”
“Tweet Location”
“Tweet Hashtags” (as a list)
“User Full Name”
“User Screen Name”
“User Location”
“User Language”
“User Profile Image link”

So i’m not sure if this can be done with this TOX, i thought it would be as easy as defining the additional parameter and column but it seems it’s more complex than that.

I’m using 0.88 build 62370 btw.

I can take a look at the additional field tomorrow, but if you’re filling in custom parameters, they should save with the project. If you change anything inside the TOX, you need to resave the TOX, which you can do by right clicking on it and clicking Save External Component (or something like that, I don’t remember the exact name) and saving over the existing TOX file. This will keep internal changes, but custom parameters you assign should save with the project.

Small note, often you’ll be using node storage for accessing things throughout the session of the toe, but you don’t actually want the value preserved when saving/loading.

In that case, use:

OP.storeStartupValue(name, value).

You can supply a dummy None value for example, instead of having TouchDesigner trying (and failing) to save and restore a very session specific object.

Ive updated some more info here:

derivative.ca/wiki099/index … ss#Storage

Cheers
Rob.

Thanks, Rob, I’ll look at updating with that :slight_smile:

cstick - can you send me link to reference API docs about the media object? I tried finding it quickly but didn’t see the key you’re looking for. This would be a sample of the Twitter object returned for tweets:

dev.twitter.com/rest/reference/ … rch/tweets

maybe was looking at the wrong api docs ?

i found it here: dev.twitter.com/overview/api/en … er-objects

media_url as an attribute of the media entity. i got it to work with the webcomp but not with your tox.