How to create the perfect Matchmaker with Bot Framework and Cognitive Services

Hi everyone, this time I wanted to showcase some of the many capabilities of Microsoft Cognitive Services using a “cupido”   bot built with Microsoft Bot Framework .

So what is the plan? Here some ideas:

  • Leverage only Facebook as channel! Why? Because with facebook you have people already “logged in” and you can leverage the messenger profile api to retrieve automatically the user details and more importantly his facebook photo!
  • Since usually the facebook photo is an image with a face , we can use this image with Vision and Face Api to understand gender, age and bunch of other interesting info without any user interaction!
  • We can score with a custom vision model that we trained using some publicly available images if a person looks like a super model or not 😉
  • Using all this info (age, gender, makeup, sunglasses, super model or not, hair color, etc…) collected with all those calls we can decide which candidates inside our database are the right ones for our user and display the ones that are fitting according to our demo rules.

Of course at the beginning our database of profiles will be empty , but with help of friends / colleagues we can quickly fill it and have fun during the demo.

So in practice how does it look like?

Here the first interaction, after saying hello the bot immediately personalizes the experience with our facebook data (foto, name, last name,etc..) and asks if we want to participate to the experiment:

After accepting it uses the described APIs to understand the image and calculate age, hair, super model score, etc…

Yeah, I know my super model score is not really good, but let’s see if there are any matches for me anyway….

Of course the bot is smart enough to display the profile of my wife otherwise I was really in a big problem :-).

Now I guess many of you have this question: how the super model score is calculated?

Well I trained the custom vision service of Microsoft with 30+ photos of real models and 30+ photos of “normal people” and after 4 iterations I had already a 90% accuracy on detecting super models in photos 😉

Of course there are several things to consider here:

  1. Images should be the focus of the picture
  2. have sufficiently diverse images, angles, lighting, and backgrounds
  3. Train with images that are similar (in quality) to the images that will be used in scoring

And we have for sure super model pics that have larger resolution, better lighting and good exposure vs the photos of “normal” people like you and me, but for the purposes of this demo the results were very good.

Another consideration to do is that you don’t always have to use Natural Language Processing in the bots (in our case in fact we skipped the usage of LUIS ) because, especially if we are not developing a Q&A/support bot, users prefer buttons and minimal amount of info to provide.

Imagine a Bot that handles your Netflix subscription, you just want  buttons like  activate/deactivate membership (if you go in vacation) and the other is “recommendations for tonight” .

Another important thing to consider is Bot Analytics and understand how your bot is performing, I leverage this great tool that under the covers uses Azure Application Insights:

If instead you are in love with statistics you can try this jupyter notebook with the following template to analyze with your custom code the Azure Application Insights metrics and events.

If you want to try the bot already with all the telemetry setup done , you can grab , compile and try the demo code (do not use this code in any production environment) that is available here and if this is your first bot start from this tutorial to understand a bit the various pieces needed.


The Customer Paradigm Executive

Fun article this time ! 

Contact me /comment if you want to apply 😂😂😂 for the role!

Customer Paradigm Executive

Candidates must be able to mesh e-business users and grow proactive networks and evolve 24/7 architectures as well.

This position requires to syndicate innovative infomediaries and morph them with viral metrics in order to synergize back-end convergence.

The objective is to deliver brand wearable eyeballs and architect granular eyeballs deployed to B2B markets incentivizing leading-edge e-business and disintermediating 24/7 relationships.

You will also have to aggregate revolutionary communities and whiteboard end-to-end systems in order to orchestrate dynamic convergence and effectively monetize efficient interfaces that can morph scalable e-markets.

Send Emails with Adobe Campaign Marketing Cloud (Neolane)

Hi this time instead of downloading data from Neolane or updating recipients in it , we want to use Neolane as email cannon leveraging its powerful template engine to manage the email look & feel and deciding on the fly ,using Neolane API, the targets of our mass email campaign.

So the use case is the following : define in Neolane an email template with some fields mapping and leverage Neolane API to send email using this template but defining the email recipients externally and also the contents of the mapped fields .

According to the official Adobe documentation this can be done using the Neolane Business Oriented Apis (we looked into the Data Oriented Apis in our previous articles) as specified here:

Unfortunately the documentation is not really clear/complete and I had really to dig inside adobe logs, error codes and soap responses to have this working properly, and here is some sample code that can help you.

The code is made using the sample provided inside the Adobe documentation (external file mapping with data coming from the CDATA section inside the delivery xml tag structure).

Here the c# code:

 string sessionToken = "Look at other Neolane Code samples on how retrieve the session token";
 string securityToken = "Look at the other Neolane Code samples on how retrieve the security token";

string scenario ="Write here the internal name of the delivery template";

 HttpWebRequest reqData = (HttpWebRequest)WebRequest.Create(adobeApiUrl);

 reqData.ContentType = "text/xml; charset=utf-8";
 reqData.Headers.Add("SOAPAction", "nms:delivery#SubmitDelivery");
 reqData.Headers.Add("X-Security-Token", securityToken);
 reqData.Headers.Add("cookie", "__sessiontoken=" + sessionToken);
 reqData.Method = "POST";


 string strWriteHeader = "<?xml version='1.0' encoding='ISO-8859-1'?>" +
 "<soapenv:Envelope xmlns:soapenv=\"\" xmlns:urn=\"urn:nms:delivery\">" +
 "<soapenv:Header/>" + 
 " <soapenv:Body>"+
 "<urn:SubmitDelivery>" +
 "<urn:sessiontoken>" + sessionToken + "</urn:sessiontoken>" +
 " <urn:strScenarioName>" +scenario+ "</urn:strScenarioName>"+

 string strWriteRecipientBody = "<delivery> " +
   "<targets fromExternalSource=\"true\"> " +
           "<externalSource><![CDATA[MsgId|ClientId|Title|Name|FirstName|Mobile|Email|Market_segment|Product_affinity1|Product_affinity2|Product_affinity3|Product_affinity4|Support_Number|Amount|Threshold " + 
"1|000001234|M.|Phulpin|Hervé|0650201020||1|A1|A2|A3|A4|E12|120000|100000]]></externalSource>" +
          "</targets> " +
 string strWriteFooter = " </urn:elemContent>" +
 "</urn:SubmitDelivery>" +
 "</soapenv:Body>" +

 string bodyData = strWriteHeader + strWriteRecipientBody + strWriteFooter;

 byte[] byteArrayData = Encoding.UTF8.GetBytes(bodyData);

 // Set the ContentLength property of the WebRequest.
 reqData.ContentLength = byteArrayData.Length;
 // Get the request stream.
 Stream dataStreamInputData = reqData.GetRequestStream();
 // Write the data to the request stream.
 dataStreamInputData.Write(byteArrayData, 0, byteArrayData.Length);
 // Close the Stream object.

 var responseData = reqData.GetResponse();

 Stream dataStreamData = responseData.GetResponseStream();
 // Open the stream using a StreamReader for easy access.
 StreamReader readerData = new StreamReader(dataStreamData);
 // Read the content.
 string responseFromServerData = readerData.ReadToEnd();

 // Clean up the streams and the response.

return responseFromServerData;

Tips on cloud solutions integration with Azure Logic Apps

The cloud paradigm is a well established reality: old CRM systems are now replaced by salesforce  solutions, HR and finance systems by workday , old exchange servers and SharePoint intranets by office 365, without mentioning entire datacenters migrated to Amazon. At the same time all the efforts that on premise where done to have all these systems to communicate together (the glorious days  of ESB solutions with TIBCO, Websphere, Bea Logic, BizTalk, etc..) seems kind of lost in the new cloud world. Why?

Well each cloud vendor (salesforce first I think) tries to position his own AppStore (AppExchange) with pre-built connectors or apps that quickly allow to companies to have integrations with other cloud platforms in a way that is not only cost effective but most of all supported by app vendor and not by a system integrator (so paid by the license).

Well this in theory should work , in practice we see a lot of good will from niche players on these apps but no or little commitment from big vendors.

Luckily however the best cloud solutions already provide rich and secure APIs to enable integration , it’s only a matter of connecting the dots and here several “integration” cloud vendors are already positioning themselves : Informatica Cloud, Dell, SnapLogic,MuleSoft,etc… ,the Gartner quadrant for Integration platform as a service (iPaaS) represents well the situation.

But while Gartner produced the report on March 2015 , Microsoft released a new kind of app on the azure platform that is called Azure Logic App.

What is the strength of this “service” compared to the others? Well in my opinion is that lies on a “de facto” proven and standard platform that can scale as much as you want and it also gives you the possibility of writing in your own connectors as Azure API apps , finally you can create your integration workflows right in the browser “no client attached!”. Of course it has so many other benefits but for me these three are so compelling that I decided to give it a try and start developing some POCs around it.

What you can do with them? Basically you can connect any system with any other system right from your browser!

Do you want to twit something if a document is uploaded on your dropbox? You can do it!

You want to define a complex workflow that start from a message on the service bus , calls a BAPI on SAP and ends inside a oracle stored procedures? It’s right there!

There is a huge list of connectors that you can use , and each of them can be combined with others in so many interesting ways!

Connectors have actions and triggers! Triggers , as the word says, are useful to react to an event (a new tweet with a specific word , a new lead on salesforce, etc..) and they can be used in a push or pull fashion (I’m interested in this event so the connector will notify me when this happens or I’m interested in this data and I will periodically call the connector to check if there is new data).

Actions are simply methods that can be executed on the connector (send an email, do a query, write a post on facebook,etc…).

An azure logic app is a workflow where you combine all these connectors using triggers and actions to achieve your goals.

How they communicate each other? I mean how do I refer inside a connector B that is linked to A to perform the action using A data? It is super simple! When you link two connectors you will see inside the target one on every action that requires data pick lists where you can easily pick the source data! This can happen because each connector automatically describes its API schema using swagger (this really rocks!).

And you want to know the best of this? If you write your own connector with Visual Studio it will automatically generate the swagger metadata for you! So in really 10 min you can have your brand new connector ready to use !

Added bonus : you can have automatically done for you a testing api made by swagger!

Azure website is full of references to quickly ramp up on this technology , so I want to give you some  useful tips in your app logic journey instead of a full tutorial.

Tip 1:  You will see that published connectors are requesting you some configuration values (Package Settings) and only after that the connector becomes usable in your logic app. I tried with no success to do the same in visual studio with a custom API app and the best that I was able to find is that you simulate this only if you use deploy script from github (look at azuredeploy.json file, some examples here ) , at this stage in fact with the deploy setup screen you can set some configuration values that will never change once your azure api app is published . The way this is done is to map deploy parameters with app settings like this:

"properties": {
"name": "[parameters('siteName')]",
"gatewaySiteName": "[parameters('gatewayName')]",
"serverFarmId": "[resourceId('Microsoft.Web/serverfarms', parameters('svcPlanName'))]",
 "siteConfig": {
  "appSettings": [
   "name": "clientId",
   "value": "[parameters('MS_client_id')]"
   "name": "clientSecret",
   "value": "[parameters('MS_client_secret')]"

Then you can use the usual ConfigurationManager.Appsettings to read these values into the code.

I guess this will be fixed (possibility of defining package settings) when the publishing on marketplace will be available.

Tip 2: If you store credentials inside your custom api app please note that by default api app are published as public…. so if this particular api app reads from you health IOT device anyone in the world that knows or discovers API address can call this API and read your health data! So set security to internal and not public!

Tip 3: Browser Designer can be sometimes instable and produce not exactly what you were hoping from it, always check also the code view!
Tip 4: Azure API Apps have application settings editable on azure portal like the normal Azure “Web” Apps but they are hidden!
Look at this blog that saved me!

That’s it!