UPDATE July 7, 2016: To simplify getting the Spark Army Knife code, I have established a new code repository that will always mirror what is running at https://spark-army-knife.appspot.com. You can thus easily download the entire Army Knife app, including the right version of the ActingWeb library, from the download section of the repository.
This post covers the code behind the Spark Army Knife. It is built on the ActingWeb library, and my previous post introduced the library and how it is used to build simple, cloud-hosted apps and how to use it for OAuth services like Cisco Spark. You don’t have to understand the ActingWeb library to create your own Spark app, just have a look at the Quick Start at the end of the previous post, and then read on here!
This post covers the code that is specific to Spark and if you have the ActingWeb library already installed, you here get the rest to set up your own Spark Army Knife! Both the ActingWeb library and the Spark code here in this post are licensed under the Apache 2.0 license.
Here is a summary of the code pieces used to implement the Spark Army Knife functionality:
- The ActingWeb library as introduced in my previous post (running on top of Google AppEngine, see my post on the Spark demo if you want to get started with AppEngine)
- An actingweb/config.py file with OAuth configurations for your Spark app
- A Spark library, spark.py, implementing the Spark API requests, as well as storing and retrieving room and message info needed
- An index.yaml file to index the Spark data stored by spark.py in the Google ndb database
- Four simple html templates for the sign-up and /makepublic functionality
- Spark-specific code in each of the four on_aw/ handlers:
on_aw_callbacks.py,
on_aw_delete.py,
on_aw_oauth.py, and
on_aw_www_paths.py
Step #1, ActingWeb Library
You should have the library already (get it from https://bitbucket.org/gregerw/acting-web-gae-library), so let’s have a look at the config.py file. (If you just want to download the ActingWeb code, there is a Download link at the left-hand side in the navigation bar.)
Step #2, config.py
Here is the init part that is relevant with <EDIT – description of what to change> where you need to make changes:
self.ui = True # Turn on the /www path # Use basic auth for /www path with creator and passphrase set/generated # when actor is created self.www_auth = "oauth" # URI for this app's actor factory with slash at end self.root = “https://<EDIT:yourappname>.appspot.com/“ self.type = "urn:actingweb:<EDIT: your_domain_or_unique_identifier:yourappname“ # The type of this actor # A human-readable description for this specific actor self.desc = “<EDIT: Human readable short description of what this app does for a user>: " self.version = "1.0" # A version number for this app self.info = “<EDIT: link to more info about your app>“ # Where can more info be found self.aw_version = "0.9" # This app follows the actingweb specification specified self.aw_supported = "" # This app supports the following options self.aw_formats = "json" # These are the supported formats self.logLevel = logging.INFO # Change to WARN for production, DEBUG for debugging, and INFO for normal testing # Hack to get access to GAE default logger logging.getLogger().handlers[0].setLevel(self.logLevel) self.auth_realm = “<EDIT: yourapp.appspot.com>” self.oauth = { # An empty client_id turns off oauth capabilities 'client_id': “<EDIT: your app’s client_id from developer.ciscospark.com>“, 'client_secret': "<EDIT: your app’s client_secret from developer.ciscospark.com>", 'redirect_uri': "https://<EDIT:yourappname>.appspot.com/oauth", 'scope': "spark:people_read spark:rooms_read spark:rooms_write spark:memberships_read spark:memberships_write spark:messages_write spark:messages_read", #<EDIT: change scope if you want a different scope for your app> 'auth_uri': "https://api.ciscospark.com/v1/authorize", 'token_uri': "https://api.ciscospark.com/v1/access_token", 'response_type': "code", 'grant_type': "authorization_code", 'refresh_type': "refresh_token", }
That’s it! You should now be able to deploy your app, go to the root URL, create a new actor, and go through the OAuth process to authorise your app to access your Spark account.
Step #3, Spark Library
However, you don’t have any Spark-specific features yet, so let’s move onto the Spark code. In a sub-directory of your app named spark/, add __init__.py with one line in it:
__all__ = ["ciscospark”]
Then add the following ciscospark.py code:
__all__ = [ 'ciscospark', ] import uuid from google.appengine.ext import ndb class Room(ndb.Model): actorId = ndb.StringProperty(required=True) id = ndb.StringProperty(required=True) title = ndb.TextProperty() sipAddress = ndb.StringProperty() webhookId = ndb.StringProperty() uuid = ndb.StringProperty() class Message(ndb.Model): actorId = ndb.StringProperty(required=True) id = ndb.StringProperty(required=True) roomId = ndb.StringProperty(required=True) personId = ndb.StringProperty() personEmail = ndb.StringProperty() date = ndb.DateTimeProperty(auto_now_add=True) class Person(ndb.Model): actorId = ndb.StringProperty(required=True) id = ndb.StringProperty(required=True) email = ndb.StringProperty(required=True) displayName = ndb.StringProperty() nickname = ndb.StringProperty() avatar = ndb.StringProperty() # This class relies on an actingweb oauth object to use for sending oauth data requests class ciscospark(): def __init__(self, oauth, actorId): self.actorId = actorId self.oauth = oauth self.spark = { 'me_uri': "https://api.ciscospark.com/v1/people/me", 'room_uri': "https://api.ciscospark.com/v1/rooms", 'message_uri': "https://api.ciscospark.com/v1/messages", 'webhook_uri': "https://api.ciscospark.com/v1/webhooks", 'person_uri': "https://api.ciscospark.com/v1/people", 'membership_uri': "https://api.ciscospark.com/v1/memberships", } def lastResponse(self): return { 'code': self.oauth.last_response_code, 'message': self.oauth.last_response_message, } def getMe(self): return self.oauth.getRequest(self.spark['me_uri']) def createRoom(self, room_title): if not room_title: return False params = { 'title': room_title, } return self.oauth.postRequest(self.spark['room_uri'], params=params) def deleteRoom(self, id=None): if not id: return False return self.oauth.deleteRequest(self.spark['room_uri'] + '/' + id) def addMember(self, id=None, email=None, personId=None): if not id or (not email and not personId): return False if email: params = { 'roomId': id, 'personEmail': email, } elif personId: params = { 'roomId': id, 'personId': personId, } return self.oauth.postRequest(self.spark['membership_uri'], params=params) def postMessage(self, id=None, text='', files=None): if not id: return False params = { 'roomId': id, 'text': text, } if files: params = { 'files': files, } return self.oauth.postRequest(self.spark['message_uri'], params=params) def getMessage(self, id=None): if not id: return False return self.oauth.getRequest(self.spark['message_uri'] + '/' + id) def deleteMessage(self, id=None): if not id: return False return self.oauth.deleteRequest(self.spark['message_uri'] + '/' + id) def getMessages(self, roomId=None, beforeId=None, beforeDate=None, max=10): if not roomId: return False params = { 'roomId': roomId, 'max': max, } if beforeId: params.update({'beforeMessage': beforeId}) elif beforeDate: params.update({'before': beforeDate}) results = self.oauth.getRequest(self.spark['message_uri'], params=params) return results['items'] def getRooms(self, max=50, uri=None): if uri: params = None else: uri = self.spark['room_uri'] params = { 'max': max, } results = self.oauth.getRequest(uri, params) if results: ret = { 'rooms': results['items'], 'next': self.oauth.next, 'prev': self.oauth.prev, 'first': self.oauth.first, } return ret else: return None def registerWebHook(self, name=None, target=None, resource='messages', event='created', filter=''): if not target or not name: return None params = { 'name': name, 'targetUrl': target, 'resource': resource, 'event': event, 'filter': filter, } return self.oauth.postRequest(self.spark['webhook_uri'], params=params) def unregisterWebHook(self, id=None): if not id: return None return self.oauth.deleteRequest(self.spark['webhook_uri'] + '/' + id) def roomAlreadyHooked(self, id, rooms): for room in rooms: if room.id == id: return True return False def hookAllRooms(self, callback_root): result = self.getRooms(max=200) while result: next = result['next'] rooms = result['rooms'] for room in rooms: if self.roomAlreadyHooked(room['id'], storedRooms): continue hook = self.registerWebHook(name=room['title'], target=callback_root + '?id=' + room['id'], filter='roomId=' + room['id']) if hook and hook['id']: if 'sipAddress' not in room: room['sipAddress'] = '' newroom = Room(actorId=self.actorId, id=room['id'], title=room['title'], sipAddress=room['sipAddress'], webhookId=hook['id']) newroom.put() if next: result = self.getRooms(uri=next) else: result = None return def unhookAllRooms(self): rooms = self.loadRooms() for room in rooms: self.unregisterWebHook(id=room.webhookId) room.key.delete() def addUUID2room(self, roomId): room = Room.query(Room.actorId == self.actorId, Room.id == roomId).get() if not room: return False if room.uuid: return room.uuid room.uuid = uuid.uuid5(uuid.NAMESPACE_URL, roomId.encode(encoding='ascii')).get_hex() room.put() return room.uuid def deleteUUID2room(self, roomId): room = Room.query(Room.actorId == self.actorId, Room.id == roomId).get() if room.uuid: room.uuid = '' room.put() return True return False def loadRoomByUuid(self, uuid): return Room.query(Room.actorId == self.actorId, Room.uuid == uuid).get() def processMessage(self, msg=None): if not msg: return False result = Person.query(Person.actorId == self.actorId, Person.email == msg['personEmail']).get() if result: message = Message(actorId=self.actorId, id=msg['id'], roomId=msg['roomId'], personId=msg['personId'], personEmail=msg['personEmail']) message.put() return True else: return False def loadMessages(self, email=None, nickname=None): if not email and not nickname: return False if not email: person = Person.query(Person.actorId == self.actorId, Person.nickname == nickname).get() if person: email = person.email if not email: return False return Message.query(Message.actorId == self.actorId, Message.personEmail == email).order(Message.date).fetch(9999) def clearMessages(self, email=None, nickname=None): if not email and not nickname: return False if not email: person = Person.query(Person.actorId == self.actorId, Person.nickname == nickname).get() email = person.email results = Message.query(Message.actorId == self.actorId, Message.personEmail == email).fetch(9999) for result in results: result.key.delete() def addTracker(self, email, nickname): if not email or not nickname: return False result = Person.query(Person.actorId == self.actorId, Person.email == email).get() if result: return False params = { 'email': email, } result = self.oauth.getRequest(self.spark['person_uri'], params) result = result['items'][0] if not result: return False if 'avatar' not in result: result['avatar'] = '' person = Person(actorId=self.actorId, id=result['id'], email=email, nickname=nickname, displayName=result['displayName'], avatar=result['avatar']) person.put() return True def deleteTracker(self, email): result = Person.query(Person.actorId == self.actorId, Person.email == email).get() if result: result.key.delete() msgs = Message.query(Message.actorId == self.actorId, Message.personEmail == email).fetch(9999) if msgs: for msg in msgs: msg.key.delete() return True return False def loadTrackers(self): return Person.query(Person.actorId == self.actorId).fetch(999) def loadRoom(self, id): return Room.query(Room.actorId == self.actorId, Room.id == id).get() def loadRooms(self): return Room.query(Room.actorId == self.actorId).fetch()
This code is very simple and you can add new methods to it easily when you need more functionality. First of all, it creates database classes for Room, Person, and Message, so that we can cache data. Then, there are quite a few simple methods that do something related to Spark: either use the ActingWeb oauth object to send Spark API requests or store/retrieve rooms, messages, and people in the Google Datastore.
Here is a snippet showing how you use the spark.py library:
spark_oauth = oauth.oauth(token=myself.getProperty('oauth_token').value) spark = ciscospark.ciscospark(spark_oauth, myself.id) msg = spark.getMessage(some_message_id)
Assuming you have an actor instantiated in myself (done by the ActingWeb library), you just initialise an oauth object with the token stored for this actor and then you create a spark object with that oauth object and the id of the actor as parameters. You can then call the spark methods. The example above retrieves the message with some_message_id by sending an API request to Spark.
Step #4, index.yaml
Following the steps above, we need to update the index.yaml (or add it) to the root of your app directory:
- kind: Message properties: - name: actorId - name: personEmail - name: date - kind: Room properties: - name: actorId - name: id
This is to tell Google to index the Message and Room tables so we can search efficiently.
Step #5, html templates
In templates/ directory, edit aw-root-factory.html (the sign-up page) and aw-actor-www-root.html (confirmation page) to your needs. If you are copying exactly the functionality in the Spark Army Knife, you need to add two new templates for the /makepublic functionality.
spark-joinroom.html:
<html> <body> You will be added to the Spark room <b>{{ title }}</b> by filling in your address below. If you are not already a Spark user, you will be invited to Spark. <form method="post"> <div>Your email address:<input type="text" name="email" maxlength="100" /></div> <input type="hidden" name="id" value="{{ id }}"> <div><input type="submit" value="Join room"></div> </form> </body> </html>
and spark-joinedroom.html:
<html> <body> You have been added to the Spark room <b>{{ title }}</b>. </body> </html>
Step #6, on_aw* functions
In the final step, you replace the on_aw_*.py files in the on_aw/ directory with the Spark Army Knife code below.
on_aw_oauth.py:
#!/usr/bin/env python import webapp2 import logging import time from google.appengine.ext import deferred from actingweb import actor from actingweb import oauth from actingweb import config from spark import ciscospark __all__ = [ 'check_on_oauth_success', ] def check_on_oauth_success(myself): spark_oauth = oauth.oauth(token=myself.getProperty('oauth_token').value) spark = ciscospark.ciscospark(spark_oauth, myself.id) me = spark.getMe() Config = config.config() if not me: return False currentId = myself.getProperty('oauthId') if not currentId.value: currentId.value = me['id'] if me['id'] != currentId.value: myself.deleteProperty('aw_tmp_redirect') return False myself.setProperty('oauthId', me['id']) if 'displayName' in me: myself.setProperty('displayName', me['displayName']) if 'emails' in me: myself.setProperty('email', me['emails'][0]) if 'avatar' in me: myself.setProperty('avatarURI', me['avatar']) chatRoom = myself.getProperty('chatRoomId') if not chatRoom.value: created = spark.createRoom("Army Knife Control") if created['id']: myself.setProperty('chatRoomId', created['id']) spark.postMessage( created['id'], "Hi there! Send me commands starting with /. Like /help") hook = spark.registerWebHook(name='Chatroom callback', target=Config.root + myself.id + '/callbacks/chatroom', filter='roomId=' + created['id']) if not hook: logging.info('Registration of chatroom webhook failed.') else: myself.setProperty('chatRoomHookId', hook['id']) deferred.defer(spark.hookAllRooms, Config.root + myself.id + '/callbacks/room') myself.setProperty('hookRoomsTime', str(time.time())) return True
This code is triggered after a successful OAuth authorisation, and it retrieves information about your Spark user, creates the Army Knife Control room and then registers webhooks for all your rooms. Note some specific, important functionality here: when the id is retrieved from Spark, it is compared to the actor’s oauthId. This is to ensure that any attempt at getting to another actor than yourself will be rejected.
on_aw_www_paths.py can be used to handle any https requests to /{actorId}/www. We don’t need this for the Spark Army Knife, but you could easily use the spark.loadRooms() method to retrieve a user’s rooms and list all the rooms with detailed information for each by adding a new html template. I leave this as an exercise 🙂
When an actor is deleted, the on_aw_delete_actor() function is called, so that you can clean up everything you need.. Here is on_aw_delete.py:
#!/usr/bin/env python # import cgi import wsgiref.handlers from actingweb import actor from actingweb import oauth from actingweb import config from spark import ciscospark import webapp2 from google.appengine.ext import deferred def on_aw_delete_actor(myself): spark_oauth = oauth.oauth(token=myself.getProperty('oauth_token').value) spark = ciscospark.ciscospark(spark_oauth, myself.id) spark.clearMessages(email=myself.creator) trackers = spark.loadTrackers() for tracker in trackers: spark.deleteTracker(tracker.email) webhookId = myself.getProperty('chatRoomHookId') if webhookId: spark.unregisterWebHook(webhookId.value) chatRoom = myself.getProperty('chatRoomId') if chatRoom: spark.deleteRoom(chatRoom.value) deferred.defer(spark.unhookAllRooms) return
The above code clears all the stored messages, deletes all the VIPs being tracked, unhooks the webhooks with the Spark platform, as well as deletes the Army Knife Control Room.
Finally, the meat of the functionality can be found in the on_aw_callbacks.py file just below. There is no magic, there are just two hook functions, one for GET incoming requests and one for POST requests (webhooks) from Spark. The GET function is used to offer the join Spark room functionality (/makepublic) and the POST function is used to process the form submitted to join a room, to process messages from all the rooms, as well as to process commands prefixed with /.
You are now done. Deploy your app to Google AppEngine!
on_aw_callbacks.py:
#!/usr/bin/env python # from actingweb import actor from actingweb import oauth from actingweb import config from spark import ciscospark from google.appengine.ext import deferred import logging import json import os import time from google.appengine.ext.webapp import template import on_aw_delete __all__ = [ 'on_post_callbacks', 'on_get_callbacks', ] def on_get_callbacks(myself, req, name): spark_oauth = oauth.oauth(token=myself.getProperty('oauth_token').value) spark = ciscospark.ciscospark(spark_oauth, myself.id) if name == 'joinroom': uuid = req.request.get('id') room = spark.loadRoomByUuid(uuid) if not room: req.response.set_status(404) return template_values = { 'id': uuid, 'title': room.title, } template_path = os.path.join(os.path.dirname(__file__), '../templates/spark-joinroom.html') req.response.write(template.render(template_path, template_values).encode('utf-8')) return def on_post_callbacks(myself, req, name, auth=None): Config = config.config() spark_oauth = oauth.oauth(token=myself.getProperty('oauth_token').value) spark = ciscospark.ciscospark(spark_oauth, myself.id) logging.debug("Callback body: " + req.request.body.decode('utf-8', 'ignore')) chatRoomId = myself.getProperty('chatRoomId').value lastHook = myself.getProperty('hookRoomsTime').value if not lastHook: lastHook = 0.0 else: lastHook = float(lastHook) now = time.time() if lastHook + (24 * 3600) < now: deferred.defer(spark.hookAllRooms, Config.root + myself.id + '/callbacks/room') myself.setProperty('hookRoomsTime', str(now)) # non-json POSTs to be handled first if name == 'joinroom': uuid = req.request.get('id') email = req.request.get('email') room = spark.loadRoomByUuid(uuid) if not spark.addMember(id=room.id, email=email): spark.postMessage(chatRoomId, "Failed adding new member " + email + " to room " + room.title) req.response.set_status(500) else: spark.postMessage(chatRoomId, "Added new member " + email + " to room " + room.title) template_values = { 'title': room.title, } template_path = os.path.join(os.path.dirname( __file__), '../templates/spark-joinedroom.html') req.response.write(template.render(template_path, template_values).encode('utf-8')) return True # Handle json POSTs below body = json.loads(req.request.body.decode('utf-8', 'ignore')) data = body['data'] responseRoomId = chatRoomId if name == 'room': callback_id = req.request.get('id') if responseRoomId == data['roomId']: req.response.set_status(204) return True if data['personEmail'] == myself.creator: name = 'chatroom' responseRoomId = data['roomId'] else: spark.processMessage(data) req.response.set_status(204) return True if name == 'chatroom': msg = spark.getMessage(data['id']) if not msg or 'text' not in msg: err = spark.lastResponse() logging.info("Error(" + str(err['code']) + " -" + err['message'] + ") in getting message from spark callback. Token may be invalid.") refresh = spark_oauth.oauthRefreshToken(myself.getProperty('oauth_refresh_token').value) if refresh: myself.setProperty('oauth_token', refresh['access_token']) myself.setProperty('oauth_token_expiry', str(now + refresh['expires_in'])) if 'refresh_token' in refresh: myself.setProperty('oauth_refresh_token', refresh['refresh_token']) myself.setProperty('oauth_refresh_token_expiry', str( now + refresh['refresh_token_expires_in'])) msg = spark.getMessage(data['id']) if not msg or 'text' not in msg: logging.warn( "Error in getting message from spark callback. Was not able to refresh token.") req.response.set_status(403) return False else: req.response.set_status(403) return False msg_list = msg['text'].lower().split(" ") if msg_list[0] == '/help': spark.postMessage(responseRoomId, "Spark Army Knife (author: Greger Wedel)\r\n\r\nUse /track email nickname to track messages from a person/VIP\r\nUse /trackers to list tracked emails.\r\nUse /untrack email to stop tracking a person.\r\nUse /get nickname to get a list of all messages since last time for that person (and /get all to get from all tracked people).\r\nUse /myurl to get the link to where your Spark Army Knife bot lives.\r\nYou can also use these commands from any room. In addition, you can use /makepublic to get a URL people can use to add themselves to a room. Use /makeprivate to disable this URL again and make the room private.\r\nUse /pin or /pin x to pin the previous message in a room (or the message x messages back). The pinned message will be listed in the Army Knife Control room.\r\n/delete DELETENOW will delete your Spark Army Knife account, this room, and all data associated with this account.") elif msg_list[0] == '/track': if len(msg_list) < 3: spark.postMessage(responseRoomId, "Usage: /track email nickname") added = spark.addTracker(msg_list[1], msg_list[2]) if added: spark.postMessage(responseRoomId, "Added tracking of " + msg_list[1]) else: spark.postMessage(responseRoomId, "Was not able to add tracking of " + msg_list[1]) elif msg_list[0] == '/myurl': spark.postMessage(responseRoomId, Config.root + myself.id + '/www') elif msg_list[0] == '/delete': if len(msg_list) == 2 and msg_list[1] == 'deletenow': on_aw_delete.on_aw_delete_actor(myself) myself.delete() else: spark.postMessage(responseRoomId, "Usage: /delete DELETENOW") elif msg_list[0] == '/pin': if len(msg_list) == 2: nr = int(msg_list[1]) - 1 else: nr = 0 if nr > 10: max = nr else: max = 10 msgs = spark.getMessages(roomId=responseRoomId, beforeId=data['id'], max=max) spark.postMessage( chatRoomId, "Pinned (" + msgs[nr]['created'] + ") from " + msgs[nr]['personEmail'] + ": " + msgs[nr]['text']) spark.deleteMessage(data['id']) elif msg_list[0] == '/get': if len(msg_list) == 2 and msg_list[1] == 'all': trackers = spark.loadTrackers() nicknames = [] for tracker in trackers: nicknames.append(tracker.nickname) else: nicknames = [msg_list[1]] for nick in nicknames: msgs = spark.loadMessages(nickname=nick) if not msgs: spark.postMessage(responseRoomId, 'No messages from ' + nick) else: spark.postMessage(responseRoomId, '-----------------------------------------') spark.postMessage(responseRoomId, 'Messages from: ' + nick) for msg in msgs: text = spark.getMessage(msg.id)['text'] room = spark.loadRoom(msg.roomId) spark.postMessage(responseRoomId, msg.date.strftime( '%c') + ' - (' + room.title + ')' + '\r\n' + text) spark.clearMessages(nickname=nick) elif msg_list[0] == '/hookall': deferred.defer(spark.hookAllRooms, Config.root + myself.id + '/callbacks/room') spark.postMessage(responseRoomId, "Started hooking into all rooms.") elif msg_list[0] == '/unhookall': deferred.defer(spark.unhookAllRooms) spark.postMessage(responseRoomId, "Started unhooking all rooms.") elif msg_list[0] == '/trackers': trackers = spark.loadTrackers() if not trackers: spark.postMessage(responseRoomId, 'No people are tracked.') for tracker in trackers: spark.postMessage(responseRoomId, tracker.email + '(' + tracker.nickname + ')') elif msg_list[0] == '/untrack': if spark.deleteTracker(msg_list[1]): spark.postMessage(responseRoomId, "Untracked " + msg_list[1]) else: spark.postMessage(responseRoomId, "Failed untracking of " + msg_list[1]) elif msg_list[0] == '/makepublic': uuid = spark.addUUID2room(responseRoomId) if not uuid: spark.postMessage(responseRoomId, "Failed to make room public") else: spark.postMessage(responseRoomId, "Public URI: " + Config.root + myself.id + '/callbacks/joinroom?id=' + uuid) elif msg_list[0] == '/makeprivate': if not spark.deleteUUID2room(responseRoomId): spark.postMessage(responseRoomId, "Failed to make room private") else: spark.postMessage( responseRoomId, "Made room private and add URL will not work anymore.") req.response.set_status(204) return True