Tweet map set up and configuration


#1

Hi, am interested in twitter monitoring, From the tweet map demo,
i would first like to know if Tweetmap is a product ?
Does it use immerse or core ?
Can i locally do the set up of the same ?


#2

Hi

Tweetmap is not a separate product it is a demonstraton of a custom application built using MapD components.

It uses MapD Core, MapD Charting and MapD Connection components.

It is currently being tidied up and modified so we can share it as a more complex open sources example. This should be complete in the next month. Once that is completed you would be able to run that code locally.

Regards


#3

Code in review, looking to make this a well-written example and expecting that others might clone it.


#4

Thank you, much appreciated for your feedback. am really looking forward to run the code locally.


#5

Hi there Jamari, as of yesterday, we made the repo for tweetmap public! You can check out the source here: Tweetmap Repo

The project is still evolving and we’re not quite ready to deploy on our main site, but should capture the idea for how to build a custom viz using our libraries. Let us know if you have trouble setting things up.


#6

Thanks alot, i have been able to do the set up, unfortunately the data is not real time as per the demos. Kindly advice if its a setting i have missed ?


#7

Hi,

The Tweetmap example is a code example.

By default it is pointing to a small static tweet dataset, there is no data streamed to that particular dataset.

regards


#8

Hi, can you please explain the process of streaming the data, so that i can have a dynamic tweet dataset ?


#9

Hi,

Depending on what you want to capture from Twitter you should look at the Twitter developer API’s or GNIP.com. These tools give you access to Twitter feeds at various scale.

Once you are gathering the data programatically you would then stream it into a MapD database using a utility like StreamInsert (docs here ).

The MapD table you are streaming into should be setup with a row_count option so it automatically manages keeping just the latest records.

regards


#10

Hi, thanks alot for the assistance and clarifications. I have managed to stream insert, but my points are not displaying on the map, despite pointing tweetmap to my db and table. It show the number of tweets but its not displaying. Maybe it might be the structure of my table … please clarify on this or share the table structure i require so that it displays…


#11

Hi

our full schema is

CREATE TABLE tweets_2017_may (
tweet_id TEXT ENCODING NONE,
tweet_time TIMESTAMP ENCODING FIXED(32),
lat FLOAT,
lon FLOAT,
sender_id BIGINT,
sender_name TEXT ENCODING DICT(32),
location TEXT ENCODING DICT(32),
source TEXT ENCODING DICT(16),
reply_to_user_id BIGINT,
reply_to_tweet_id BIGINT,
lang TEXT ENCODING DICT(8),
followers INTEGER,
followees INTEGER,
tweet_count INTEGER,
join_time TIMESTAMP ENCODING FIXED(32),
tweet_text TEXT ENCODING NONE,
country TEXT ENCODING DICT(8),
admin1 TEXT ENCODING DICT(32),
admin2 TEXT ENCODING DICT(16),
place_name TEXT ENCODING DICT(32),
state_abbr TEXT ENCODING DICT(8),
county_state TEXT ENCODING DICT(16),
origin TEXT ENCODING DICT(8),
hashtags TEXT[] ENCODING DICT(32),
tweet_tokens TEXT[] ENCODING DICT(32),
goog_x FLOAT,
goog_y FLOAT,
is_exact BOOLEAN)

but you do not need all of this for the tweetmap app

i think

CREATE TABLE tweets_small (
tweet_id TEXT ENCODING NONE,
tweet_time TIMESTAMP ENCODING FIXED(32),
lat FLOAT,
lon FLOAT,
sender_name TEXT ENCODING DICT(32),
lang TEXT ENCODING DICT(8),
tweet_text TEXT ENCODING NONE,
origin TEXT ENCODING DICT(8),
hashtags TEXT[] ENCODING DICT(32),
tweet_tokens TEXT[] ENCODING DICT(32)
)

is probably enough for the app

regards


#12

Thanks alot, for points to appear on the Tweetmap do we require GPU, am asking this because despite having lat and Lon the points are not displaying…


#13

Hi

Yes, for backend render a GPU is required.

Regards


#14

Hi,
Have created a table in MapD database so as to stream insert twitter data, for use by Tweetmap application.
Kindly help me know which twitter object properties maps to the tweets table fields?

Below is the table schema.
CREATE TABLE tweets_small (
tweet_id TEXT ENCODING NONE,
tweet_time TIMESTAMP ENCODING FIXED(32),
lat FLOAT,
lon FLOAT,
sender_name TEXT ENCODING DICT(32),
lang TEXT ENCODING DICT(8),
tweet_text TEXT ENCODING NONE,
origin TEXT ENCODING DICT(8),
hashtags TEXT[] ENCODING DICT(32),
tweet_tokens TEXT[] ENCODING DICT(32)
)

Regards,
Mark


#15

Hi,

Please identify what twitter object properties you have available and then we can work on the mapping,

regards