Deploying and predicting with model

This notebook illustrates:

  1. Deploying model
  2. Predicting with model
In [1]:
# change these to try this notebook out
BUCKET = 'tensorflow-20200504-013719'
PROJECT = 'qwiklabs-gcp-02-6dd5700d2ca2'
REGION = 'us-central1'
In [2]:
import os
os.environ['BUCKET'] = BUCKET
os.environ['PROJECT'] = PROJECT
os.environ['REGION'] = REGION
os.environ['TFVERSION'] = '1.13'  
In [3]:
%%bash
if ! gsutil ls | grep -q gs://${BUCKET}/babyweight/trained_model; then
  gsutil mb -l ${REGION} gs://${BUCKET}
  # copy canonical model if you didn't do previous notebook
  gsutil -m cp -R gs://cloud-training-demos/babyweight/trained_model gs://${BUCKET}/babyweight/trained_model
fi
Creating gs://tensorflow-20200504-013719/...
Copying gs://cloud-training-demos/babyweight/trained_model/eval/events.out.tfevents.1529348264.cmle-training-master-a137ac0fff-0-9q8r4...
Copying gs://cloud-training-demos/babyweight/trained_model/checkpoint...
Copying gs://cloud-training-demos/babyweight/trained_model/export/exporter/1529355466/saved_model.pb...
Copying gs://cloud-training-demos/babyweight/trained_model/events.out.tfevents.1529347276.cmle-training-master-a137ac0fff-0-9q8r4...
Copying gs://cloud-training-demos/babyweight/trained_model/export/exporter/1529355466/variables/variables.data-00000-of-00001...
Copying gs://cloud-training-demos/babyweight/trained_model/export/exporter/1529355466/variables/variables.index...
Copying gs://cloud-training-demos/babyweight/trained_model/graph.pbtxt...       
Copying gs://cloud-training-demos/babyweight/trained_model/model.ckpt-342784.data-00000-of-00003...
Copying gs://cloud-training-demos/babyweight/trained_model/model.ckpt-342784.data-00001-of-00003...
Copying gs://cloud-training-demos/babyweight/trained_model/model.ckpt-376661.data-00002-of-00003...
Copying gs://cloud-training-demos/babyweight/trained_model/model.ckpt-376661.data-00000-of-00003...
Copying gs://cloud-training-demos/babyweight/trained_model/model.ckpt-342784.meta...
Copying gs://cloud-training-demos/babyweight/trained_model/model.ckpt-376661.data-00001-of-00003...
Copying gs://cloud-training-demos/babyweight/trained_model/model.ckpt-376661.index...
Copying gs://cloud-training-demos/babyweight/trained_model/model.ckpt-342784.data-00002-of-00003...
Copying gs://cloud-training-demos/babyweight/trained_model/model.ckpt-390628.data-00000-of-00003...
Copying gs://cloud-training-demos/babyweight/trained_model/model.ckpt-342784.index...
Copying gs://cloud-training-demos/babyweight/trained_model/model.ckpt-376661.meta...
Copying gs://cloud-training-demos/babyweight/trained_model/model.ckpt-390628.data-00002-of-00003...
Copying gs://cloud-training-demos/babyweight/trained_model/model.ckpt-390628.data-00001-of-00003...
Copying gs://cloud-training-demos/babyweight/trained_model/model.ckpt-390628.index...
Copying gs://cloud-training-demos/babyweight/trained_model/model.ckpt-390628.meta...
- [22/22 files][  6.5 MiB/  6.5 MiB] 100% Done                                  
Operation completed over 22 objects/6.5 MiB.                                     

Deploy trained model

Deploying the trained model to act as a REST web service is a simple gcloud call.

In [4]:
%%bash
gsutil ls gs://${BUCKET}/babyweight/trained_model/export/exporter/
gs://tensorflow-20200504-013719/babyweight/trained_model/export/exporter/1529355466/
In [5]:
%%bash
MODEL_NAME="babyweight"
MODEL_VERSION="ml_on_gcp"
MODEL_LOCATION=$(gsutil ls gs://${BUCKET}/babyweight/trained_model/export/exporter/ | tail -1)
echo "Deleting and deploying $MODEL_NAME $MODEL_VERSION from $MODEL_LOCATION ... this will take a few minutes"
#gcloud ai-platform versions delete ${MODEL_VERSION} --model ${MODEL_NAME}
#gcloud ai-platform models delete ${MODEL_NAME}
gcloud ai-platform models create ${MODEL_NAME} --regions $REGION
gcloud ai-platform versions create ${MODEL_VERSION} --model ${MODEL_NAME} --origin ${MODEL_LOCATION} --runtime-version $TFVERSION
Deleting and deploying babyweight ml_on_gcp from gs://tensorflow-20200504-013719/babyweight/trained_model/export/exporter/1529355466/ ... this will take a few minutes
WARNING: Using endpoint [https://ml.googleapis.com/]
Created ml engine model [projects/qwiklabs-gcp-02-6dd5700d2ca2/models/babyweight].
WARNING: Using endpoint [https://ml.googleapis.com/]
Creating version (this might take a few minutes)......
..........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................done.

Use model to predict (online prediction)

Send a JSON request to the endpoint of the service to make it predict a baby's weight. The order of the responses are the order of the instances.

In [6]:
from oauth2client.client import GoogleCredentials
import requests
import json

MODEL_NAME = 'babyweight'
MODEL_VERSION = 'ml_on_gcp'

token = GoogleCredentials.get_application_default().get_access_token().access_token
api = 'https://ml.googleapis.com/v1/projects/{}/models/{}/versions/{}:predict' \
         .format(PROJECT, MODEL_NAME, MODEL_VERSION)
headers = {'Authorization': 'Bearer ' + token }
data = {
  'instances': [
    {
      'key': 'b1',
      'is_male': 'True',
      'mother_age': 26.0,
      'plurality': 'Single(1)',
      'gestation_weeks': 39
    },
    {
      'key': 'g1',
      'is_male': 'False',
      'mother_age': 29.0,
      'plurality': 'Single(1)',
      'gestation_weeks': 38
    },
    {
      'key': 'b2',
      'is_male': 'True',
      'mother_age': 26.0,
      'plurality': 'Triplets(3)',
      'gestation_weeks': 39
    },
    {
      'key': 'u1',
      'is_male': 'Unknown',
      'mother_age': 29.0,
      'plurality': 'Multiple(2+)',
      'gestation_weeks': 38
    },
  ]
}
response = requests.post(api, json=data, headers=headers)
print(response.content)
b'{"predictions": [{"predictions": [7.740230083465576], "key": ["b1"]}, {"predictions": [7.247548580169678], "key": ["g1"]}, {"predictions": [6.182091236114502], "key": ["b2"]}, {"predictions": [6.13692569732666], "key": ["u1"]}]}'

The predictions for the four instances were: 7.66, 7.22, 6.32 and 6.19 pounds respectively when I ran it (your results might be different).

Use model to predict (batch prediction)

Batch prediction is commonly used when you thousands to millions of predictions. Create a file with one instance per line and submit using gcloud.

In [7]:
%%writefile inputs.json
{"key": "b1", "is_male": "True", "mother_age": 26.0, "plurality": "Single(1)", "gestation_weeks": 39}
{"key": "g1", "is_male": "False", "mother_age": 26.0, "plurality": "Single(1)", "gestation_weeks": 39}
Writing inputs.json
In [8]:
%%bash
INPUT=gs://${BUCKET}/babyweight/batchpred/inputs.json
OUTPUT=gs://${BUCKET}/babyweight/batchpred/outputs
gsutil cp inputs.json $INPUT
gsutil -m rm -rf $OUTPUT 
gcloud ai-platform jobs submit prediction babypred_$(date -u +%y%m%d_%H%M%S) \
  --data-format=TEXT --region ${REGION} \
  --input-paths=$INPUT \
  --output-path=$OUTPUT \
  --model=babyweight --version=ml_on_gcp
jobId: babypred_200503_170118
state: QUEUED
Copying file://inputs.json [Content-Type=application/json]...
/ [1 files][  205.0 B/  205.0 B]                                                
Operation completed over 1 objects/205.0 B.                                      
CommandException: 1 files/objects could not be removed.
Job [babypred_200503_170118] submitted successfully.
Your job is still active. You may view the status of your job with the command

  $ gcloud ai-platform jobs describe babypred_200503_170118

or continue streaming the logs with the command

  $ gcloud ai-platform jobs stream-logs babypred_200503_170118

Copyright 2017 Google Inc. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License