This post is part of a series. If you want to work on creating your own Wildlife ID environment, start here!
In this segment we’re creating a Logic App that brings several other components together. The end goal is to examine a picture with EXIF data, identify a wildlife in the image, copy it to Azure storage, extract the EXIF GPS data and store the results in Cosmos DB. The basic architecture of our Logic App is the following.
The Logic App overview looks like this:
So, let’s get started! Start by navigating to your Azure Portal and provisioning the Logic App.
Select to create your Logic App and provisioning will start. Select to create a blank Logic App.
Add a first step of OneDrive, when File is Created
Then, sign in and select “Yes” to provide access to the services. Then, configure your OneDrive as follows.
Add the next step of “Get File Content Using Path”
Add the “file path”
Add a next step of “create blob”
Create your connection and name it with an attachment to the storage account we provisioned earlier.
Configure your folder path for /wildlife (that we created in the storage account earlier) and then connect to the blob and blob content.
Then, add HTTP as the next step. In this step, you are requesting the Microsoft AI service to identify the wildlife. In order to use it, you need to request access with a key from the AI for Earth team. If you are doing a custom model, you could also plug in your own model here, or attach to Cognitive Services if you just wanted to do basic object recognition. The difference between using the AI for Earth models is “this is a bird” vs. “this is a red headed woodpecker”… plus a bunch of other details. If you just want to do object detection, start with Cognitive Services.
We then progress into getting the GPS information from our Azure Function that we created earlier. Add another HTTP step and do the following:
Then, continue with the parse JSON step to grab the output of the Azure Function and capture it for storage into the Cosmos DB.
You can get the JSON in a couple ways. One way, is to use the “use sample payload to generate schema” button. You’ll insert the output of your test run against the Azure Function to create the schema.
I’ve also included the schema below:
{
"properties": {
"lat": {
"type": "number"
},
"lon": {
"type": "number"
}
},
"type": "object"
}Now you’ll have a second parse JSON, this time from the output from the AI model output. This will turn the returned information from the AI model output into JSON we’ll use later.
Here is the JSON schema for that:
{
"properties": {
"bboxes": {
"items": {
"properties": {
"confidence": {
"type": "number"
},
"x_max": {
"type": "number"
},
"x_min": {
"type": "number"
},
"y_max": {
"type": "number"
},
"y_min": {
"type": "number"
}
},
"required": [
"confidence",
"x_max",
"x_min",
"y_max",
"y_min"
],
"type": "object"
},
"type": "array"
},
"predictions": {
"items": {
"properties": {
"class": {
"type": "string"
},
"class_common": {
"type": "string"
},
"confidence": {
"type": "number"
},
"family": {
"type": "string"
},
"family_common": {
"type": "string"
},
"genus": {
"type": "string"
},
"genus_common": {
"type": "string"
},
"kingdom": {
"type": "string"
},
"kingdom_common": {
"type": "string"
},
"order": {
"type": "string"
},
"order_common": {
"type": "string"
},
"phylum": {
"type": "string"
},
"phylum_common": {
"type": "string"
},
"species": {
"type": "string"
},
"species_common": {
"type": "string"
},
"subphylum": {
"type": "string"
},
"subphylum_common": {
"type": "string"
},
"subspecies": {
"type": "string"
},
"subspecies_common": {
"type": "string"
}
},
"required": [
"class",
"class_common",
"confidence",
"family",
"family_common",
"genus",
"genus_common",
"kingdom",
"kingdom_common",
"order",
"order_common",
"phylum",
"phylum_common",
"species",
"species_common",
"subphylum",
"subphylum_common"
],
"type": "object"
},
"type": "array"
}
},
"type": "object"
}You’ll now create a series of For-Each tied to predictions and bboxes coming out of the JSON output:
For “create or update document” we’ll be storing the output from all this into the Cosmos DB. There is a lot of attaching to do. Make sure you don’t forget the partition key at the bottom. You’ll need to add the variable and attach it, which I’ve tied to species_common (you’ll recall this from the earlier creation of the Cosmos DB partition key).
Here is the code output of that:
{
"blob_etag": "@{body('Create_blob')?['ETag']}",
"blob_filelocator": "@{body('Create_blob')?['FileLocator']}",
"blob_name": "@{body('Create_blob')?['DisplayName']}",
"blob_path": "@{body('Create_blob')?['Path']}",
"class": "@{items('For_each')?['class']}",
"class_common": "@{items('For_each')?['class_common']}",
"confidence": "@{items('For_each_2')?['confidence']}",
"family": "@{items('For_each')?['family']}",
"family_common": "@{items('For_each')?['family_common']}",
"file_entity": " @{triggerOutputs()['headers']['x-ms-file-etag']}",
"genus": "@{items('For_each')?['genus']}",
"genus_common": "@{items('For_each')?['genus_common']}",
"id": "@{triggerOutputs()['headers']['x-ms-file-id']}",
"image_link": "@{triggerOutputs()['headers']['x-ms-file-path-encoded']}",
"kingdom": "@{items('For_each')?['kingdom']}",
"kingdom_common": "@{items('For_each')?['kingdom_common']}",
"lat": "@{body('Parse_JSON_2')?['lat']}",
"lon": "@{body('Parse_JSON_2')?['lon']}",
"order": "@{items('For_each')?['order']}",
"order_common": "@{items('For_each')?['order_common']}",
"phylum": "@{items('For_each')?['phylum']}",
"phylum_common": "@{items('For_each')?['phylum_common']}",
"species_common": "@{items('For_each')?['species_common']}"
}Now, test it out! Success!
So… with all this combined, we’ve created something pretty cool. For several reasons:
- It’s useful
- It provides impact from an environmental awareness standpoint
- It’s a great education tool for learning to code
- It’s expandable to do new and interesting things
- You can replace components and do something completely different
Enjoy!
Nathan Lasnoski
Nathan Lasnoski 