In past articles, I shared my experience working with microservices on Google Cloud Platform: the different options we have, pros, cons and limitations of each service in GCP.
Since this is the third part of the series, please be sure to read the first and the second part in order to have a better background for what we are going to discuss in here: advice and methods to approach security in GCP's App Engine.
By default all requests to App Engine use HTTPS - this is actually very secure and is supported out of the box. With regards to the domains, you have two options:
a) The appspot.com domain for your app, which is the default option. In this case your application shall be accessible like this:
https://PROJECT_ID.REGION_ID.r.appspot.com
b) Custom Domain: Attach your own custom domain. This is something we’ll do later during this training, with this you can route your service to your company domain or using a subdomain in your own domain.
All GCP resources are secure using IAM, and you can assign different roles to all your users or developers depending on the nature of work they will do.
It's very common for the app engine applications to consume other services in GCP like cloud storage, cloud datastore, bigquery or cloudSQLs. Luckily, GCP allows you to give specific permissions to the App Engine to have access to these services using service accounts.
This is used to restrict the access to your application depending on what each team member requires to do, developers should be able to push new code to the app engine, Support team should be able to see application logs or debug your application.
App Engine allows you to specify firewall rules for your applications directly on the cloud. With this, you can allow traffic only from specific IPs, or even block specific IPs too.
With App Engine you can also restrict your application to get traffic from:
Let's recap the example we were discussing in the previous entries: a sales company wants to develop a new service that gets notifications from an external provider every time they are out of stock for a particular product.
Internally, the service should save a new registry in cloud data store for future reference.
The sales company wants our help to develop and secure the out of stock service. To avoid unauthenticated traffic and false alarms, the company asks our help to design a solution for this use case. They also want to take advantage of this enhancement to use a subdomain controlled by them (like this one).
So we need to design a solution like this.
Solution Architecture
Architecture Description
The first thing we'll be doing is configure a custom domain for our app engine service, then we will use app engine firewall rules to allow calls only from the public IP of the third party vendor. This is the best way to secure our recently exposed service over the internet.
The communication between our new service and datastore should be also secure using IAM with a specific service account with restricted access.
Solution Development
The out of stock service development is quite simple - you can find the repository right here.
In order to insert kinds into the datastore database, we need to include some dependencies, so we add this to index.js:
const {Datastore} = require('@google-cloud/datastore');
Same thing for the package.json. Here we are including some dependencies like UUID to auto-generate IDs for our kinds, the datastore of course, the debugger agent in GCP and the express for the routes of our web service.
"dependencies": { "@google-cloud/datastore": "^6.3.1", "@google-cloud/debug-agent": "^5.1.3", "check-node-version": "^4.1.0", "express": "^4.17.1", "uuid": "^8.3.2" }, In index.js, there are at least to pieces that are important, this one is where the post route is define, here what we are doing is getting the object from the call and calling the saveNotification method.
app.post('/', async (req, res) => { const notification = req.body; console.log(`Request ${notification}`) try { console.log(`Out of stock report ${notification.product.id}`); saveNotification(notification.product); console.log(`Out of stock notification saved ${notification.product.id}`); res.status(200).send(); } catch (ex) { console.log(`Out of stock notification error ${notification.product.id} failure: ${ex}`); res.status(500).send(); } }) The save notification method is where the magic happens: first of all, an auto-generated ID is created using the UUID utility. This is important because all our kinds shall have a unique identifier, then we define the kind 'notification' and create the new notification object that is saved in datastore.
function saveNotification(product) { console.log('Saving notification'); var id = uuidv4(); const kind = 'notification'; // The Cloud Datastore key for the new entity const notificationKey = datastore.key([kind, id]); const notification = { key: notificationKey, data: { product: product }, }; // [END datastore_entity_with_parent] return datastore.save(notification); } To create our database in the datastore, you can follow this tutorial, then you can deploy the solution in App Engine.
My code is directly in git so I just clone it into the cloud shell. From here you need to create the app in App Engine with the following commands:
gcloud app create (you will probably need to include the region, you can see the previous tutorial for this);rpm install;gcloud app deploy.In the output of this command you will find the link to our new service (in my case, it is this one). Now I test the service using Postman as email client.
This is the object that is accepted by the service:
{ "product": { "id": "2", "name": "laptop", "brand": "css" } } 
Everytime we call this service, a new entry will be stored in the datastore on GCP.
Custom domain configuration in App Engine
For now our service is responding to a default URL exposed by Google, in the following format https://PROJECT_ID.REGION_ID.r.appspot.com. In order to route our service to a different domains we first need to go to the settings option in the App Engine dashboard.
Here, Google will force you to verify the ownership for the domain, in my case I used a CNAME record for that purpose.
All good now:
Now we are able to add a new custom domain for our service. Be sure to add a new mapping/subdomain for the SSL and click save.
We just need to wait for the ssl to be ready for our subdomain, WHICH could take a while.
Next, we need to create the CNAME record for our service in our Cloud DNS zone:
We also need a dispatcher yaml to route traffic and make a small change in our code. To create it, we go through these steps:
nano dispatch.yaml, then add the following content:url: "services.xtiyo.com/out-of-stock*"
service: out-of-stock-service
gcloud app deploy dispatch.yaml
Now let's apply a small fix in our code to be able to route the traffic with our new dispatcher. Code should be like the following example, the only change is in the routes that now include the /out-of-stock path for the get and the post methods.
app.get('/out-of-stock', async (req, res) => { console.log(`Out Of Stock Service: health check`); res.status(200).send('OK'); }) app.post('/out-of-stock', async (req, res) => { const notification = req.body; console.log(`Request ${notification}`) try { console.log(`Out of stock report ${notification.product.id}`); saveNotification(notification.product); console.log(`Out of stock notification saved ${notification.product.id}`); res.status(200).send(); } catch (ex) { console.log(`Out of stock notification error ${notification.product.id} failure: ${ex}`); res.status(500).send(); } }) Now lets deploy the changes and that’s it:
gcloud app deploy
All is working as expected:
Adding app engine firewall rules.
Now is time to configure firewall rules to whitelist/block traffic from different or specific sources, let’s go to the firewall rules section in the app engine dashboard.
What we will do is allow access just from one specific IP. For security purposes, I'll use my public IP then click on CREATE RULE option in the firewall rules page.
We’ll only permit access from my specific public IP like this. The priority is 1 because I need this rule to be evaluated before the rest. The action on match will be 'allow' (to permit traffic) and the IP range in this case was obtained by using this link. The /32 is important as this is only one single IP.
Now let's add another rule to block all the rest of traffic. Click on the 3 dots of the default rule and the edit, change the action on match to 'deny'.
If I try to access the service from my computer, all will work as expected but it will fail from a different location.
This is the situation when I call the service from my workstation:
A 403 error happens if I call it from a different place/different public IP - Google will block the traffic.
Conclusion
App Engine is just amazing to implement microservices and secure them in a very easy way.
The firewall rules are very useful to control the access to our exposed services in a granular manner, while the use of a custom domain is also helpful to keep consistency between your services and applications at organization level.
Learned something new here? Maybe you want to learn more stuff by subscribing to the Around25 newsletter (check below). No spammy content, just solid tech bites and tutorials from our side.