Alexa Meeting Room Assistant (2)

Part 2 sees the Matchbox team get deeper into the development of a smarter office

Alexa Meeting Room Assistant (2)

Part 2 sees the Matchbox team get deeper into the development of a smarter office

If you’ve not read Part 1 in our ambitious project to create an Alexa Room Assistant, so far we’ve finalized our technical setup and defined how it should operate.

With a well-planned synthesis of Raspberry Pi 2, Microsoft IoT core, LED matrix and Amazon Echo, things are starting to take shape for the Matchbox team of Giulio Ladu (Development) and Adam Thoms (Design).

Creating a Hosted Service for Alexa (Continued)

Giulio Ladu – First thing to note when looking at hosting anything on Windows IoT core is that there is no IIS (Internet information services), which is somewhat limiting. However Microsoft do provide a solution which shows you how to create your very own server from scratch. I thought this would give me an opportunity to get back to basics; the solution I found was the Microsoft Windows IoT core Blinky Sample. In this sample there are two projects, one acts as the web server and the other acts as the IoT device. Some issues here meant was not the kind of format I needed, however it did provide a great starting point.

Once the basic server code was up and running on the Raspberry Pi, it was time to see if the Amazon passthrough would be able to interact with our server. Receiving that first request from Amazon was a massive relief, and provided a great insight into how everything would interact.

The next step was to create a library which would know how to deal with this request, and in turn format the required response. After much documentation reading and trawling through forums I came across a blog post which explained how to get started with Alexa development using .Net. Within this blog post they discuss a library which had been written to parse the request made by Alexa services and allow you to create a skill using .Net.

After reading through the code I attempted to combine the library with the already implemented server code, however I found that to deploy anything to IoT core it has to make use of the Windows 10 Universal Library. Once the .Net library was integrated into the server, I was able to receive a call from the Alexa services and generate an output using C#.

Bonus! 

Connecting Alexa to our custom server running on the Raspberry Pi worked, however there was still one more piece to the puzzle, connecting to the Arduino over Bluetooth. For this I wanted to create a custom BLE (Bluetooth Low Energy) profile that would expose a service which would allow me to send a string value over to the Arduino. Fortunately, this was something which could be designed fairly quickly and easily as members of the Matchbox team have been working closely with Bluetooth over the last few years to create a piece of software called Bluetooth Developer Studio.

BDS allows you to create a profile and see how it will interact, as well as send and receive data, with a virtual device. After designing the profile in BDS I was able to use a piece of software provided by Nordic Semiconductor to generate the files needed to convert our BLE shield to start broadcasting the correct profile. Once I had the Arduino broadcasting my custom profile I was able to go back to BDS, however instead of testing against a virtual device, I was able to test against my actual Bluetooth device! 

A Foray into Industrial Design

Adam Thoms – As our Alexa project developed and evolved; a small mountain of various Arduino parts, Bluetooth adapters, and wires began to accumulate on Giulio’s desk.

Considering our ‘real-world’ approach to this project, it was apparent the impressive yet fragile system he had created wouldn’t last too long when left in the presence of any unsupervised Matchbox rabble. It would need some sort of container, both for protection and so it could be neatly stored by the Meeting Room door.

Step one was to start jotting down the dimensions of Giulio’s hardware setup, noting the spacing each part would require from its neighbours and if any outlets where required, such as power leads or air vents. From here I sketched out a design that could contain the items while also being secured to the wall.

After sharing the initial sketch in our weekly project meeting I set to work creating a prototype. With a cursory glance into the Matchbox Recycling Box I soon found the perfect compartments. Fortunately for me somebody had recently received a large delivery of books, this being an Echo project I couldn’t have wished for a better prototyping material than Amazon branded cardboard. 

Progress with Bluetooth Connectivity

GL  With the Bluetooth profile working correctly, all I needed to do now was to ensure the Raspberry Pi was connected over Bluetooth to the Arduino. Firstly, it is worth noting that at the time of writing this there are only a limited number of Bluetooth dongles which are supported by IoT core, so I had to be sure to purchase one from the defined list. Thus, with a shiny new supported BLE dongle I went to work creating a universal app “test project” that would allow me to work on the new functionality without affecting my current server code. 

Working with Windows IoT core was quite easy in this respect, as I have worked on previous products which use Bluetooth in the past and as the IoT core can run universal applications, it was simply a case of following these steps:

  • First we query the operating system to get a list of all connected devices, filtering them on the device name (which had been defined in the creation of the BLE profile).
  • Once we have the device (in this case the Arduino) we can query it for available Gatt Services filtering on the service UUID we specified when creating our Profile.
  • Finally with the correct service we are able to query the service for available Gatt Characteristics again filtering the results on the UUID we specified when creating the profile.

Once we have the characteristic we are able to write data to the Arduino, which was the final part of the hardware puzzle. After the Bluetooth communications were complete, we were able to Ask Alexa a question, retrieve the data in our web server, format it, and then pass any data through to the Arduino. It was looking good!

Matchbox Gets 3D Printing

AT – From the perspective of the end user, the LED board is clearly the focus of their attention. This was one of my key concerns when fleshing out the design in three dimensions; as the LED board has no frame (the matrix of diodes begins and ends with a marginal plastic boarder) any part of the supporting container could easily begin to impede on the visibility of the content. The prototype phase allowed us to test how thin we could make this supporting frame while feasibly maintaining enough strength to hold the hardware inside. It also granted us the opportunity to see how each item sat inside the box, illustrating if more or less space would be required in certain areas.

With the prototype built and presented to the rest of the team, the unanswered question we pondered was ‘How do we make this for real?’.

  • Should we go down the wood-working route? Slightly archaic for such a cutting edge project.
  • Maybe metal? Too heavy duty.
  • How about Lego? Tempting but tenuous!

There was only one answer to fit the bill, with Giulio pushing the boundaries of Amazon Echo software development, far be it for the Design team to shy away from emerging technology. Despite a lack of existing CAD/CAM experience we would 3D print our storage container.

Where to start? After some internet research we discovered BuildBrighton. A local community-driven workshop equipped with 3D printers, laser cutters, routers, lathes and pretty much everything else you can think of. Following an introductory email expressing our project to BuildBrighton, Alex kindly responded with information about their open evening, a chance for any interested Joe Public to visit the workshop and talk with existing members.

On a somewhat chilly Thursday in December I eagerly ventured to BuildBrighton HQ, with my cardboard prototype in hand. There I was greeted by Felix, who gave me a tour of the premises, pointing out the wide range of tools and equipment available. After chatting with some other members about what Matchbox was hoping to achieve, I was introduced to Jon. He was able to look at the prototype model and suggest how I should go about 3D printing it.

We now had a plan (with insider knowledge of the Tuesday BuildBrighton 3D Printing night in two weeks’ time) it had three steps:

  • Sign up as a BuildBrigton member (they recommended a payment of £20 per month)
  • Create a digital 3D model of our container
  • Take the model to get printed at the workshop’s 3D Printing night

Office 365 Woes

GL – Now we had some dummy data coming into the Arduino, but we still needed a way to access our office’s shared calendar, and retrieve all meetings that were hosted in the Meeting Room that day. We discovered that accessing your own calendar was fairly straight forward, getting a calendar for a room was not so much. It appears that the ease of getting a shared calendar depends on how your Office 365 is set up, and whether you are using Azure active directory. I proved that it was possible by using Outlook COM object in a test project, however this would not work running on the Raspberry Pi as it relies on the device having an instance of Outlook running. So we knew it was possible but we were still trying to find a solution. 

EWS (Exchange Web Services) to the rescue! Again, after setting up a test project we were able to start digging around in EWS, hours of trial and error and we managed to get the data we were looking for. Success we thought, however when trying to port our written code over we were hit with another problem. The code could not be hosted on the server, as a lot of the libraries which EWS relies on do not work within a Windows 10 Universal application (which is needed to run on IoT core). This was luckily not a difficult problem to solve as we were able to create a new ASP web API and host it on Azure. After some code formatting and restructuring we now had a web API which could be called from our IoT core server, and receive a JSON file containing a list of all meetings between two dates.

From here on out this would be plain sailing, all we had to do was use the previously created library to issue a speechlet (used to define what Alexa will say, and update the Alexa card) and build a response to return, which would result in Alexa reading out our calendar data. Working with the LED matrix on the other hand would prove to be somewhat problematic.

Creating the 3D Model

AT – Having some 3D modelling experience for real-time environments and animation I opened the Autodesk Fusion 360 software that had been recommended to me without complete bewilderment and abhorrence. Still, the Design & Technology lessons of yesteryear felt like a long time ago. After much trial and error, lots of coffee and loss of patience; I arrived at a point where I was happy to call the 3D model complete and ready to attempt printing.

However, all was not to go to plan. As Tuesday night quickly approached, the 3D Printing evening at BuildBrighton had to be called off, their largest printer was temporarily inoperative. Fast approaching our deadline of Near Year and with everyone dispersing for the Christmas holidays that week, we needed to find a solution that would fit within our impending delivery date.

Fine-tuning the LED Matrix

GL  During the initial design phase we decide what we wanted the LED matrix to display, and what necessary information would be needed to send to the Arduino. This was my first large Arduino project, and I was under the illusion that any LED matrix would be compatible. I thought all we had to do was connect the Arduino, and away we go! What I didn’t take into consideration was that the Bluetooth shield we were using (despite sitting on top of the Arduino) actually needed to have access to some pins. Rookie mistake!

After everything was connected, we could see the Bluetooth device on the IoT core device – unfortunately, whenever the Arduino tried to update the matrix with some text, the Bluetooth connection would drop. Explaining it now, it kind of makes sense, however being a rookie this was a bit of a nightmare to figure out. After talking to some people on forums and discussing with members of both the Arduino forums and the Adafruit forums we managed to pinpoint the problem. The Arduino was two pins short of being able to deliver the UI we wanted.

This was due the LED matrix being made up of two lines for writable text, this would be controlled by 3 pins for the top, 1 pin for each R,G,B and the same for the bottom line. Two of the pins we needed in the bottom line were being used by the Bluetooth shield, so we could have full colour on the top line and only red on the bottom – after looking at our original designs is fairly close to what we wanted, we could make this work!

Into the Home Straight

Having come to the end of Part 2 we’ve built upon the technical approach with development time going into Windows IoT core, creating a Bluetooth profile and setting up the sharing of calendar data.

In Part 3 the project comes to its grand finale! The team ties up all loose ends, including the troublesome 3D printing aspect.

I wanted to create a custom BLE (Bluetooth Low Energy) profile. Fortunately, members of the Matchbox team have been working closely with Bluetooth over the last few years to create a piece of software called Bluetooth Developer Studio.
Giulio Ladu
Development

Highlights

  • Bluetooth Developer Studio
  • 3D Printing
  • Windows IoT Core
  • Amazon Echo
  • Raspberry Pi 2
  • Arduino
  • LED Matrix