PROG2003 – Cloud Systems Development
Assignment 2
PROG2003 Assignment 2
Weight: 40% of your final mark
Due: Week 6 Monday (5 December 2022) at 11:00 PM
Specifications
In Part A, your task is to implement a Lambda function that is invoked after a “Student.json” object is
created in an S3 bucket. Upon execution, the Lambda function reads the content in the “Student.json”
file and inserts them into a DynamoDB table. In Part B, your task also includes preparing a dataset by
applying pre-processing techniques. Your submission will be evaluated based on:
• whether instructions have been followed;
• co
ect functionality;
• co
ect implementation; and
• comments inside the program
Getting Help
This assignment, which is to be completed individually, is your chance to gain an understanding of the
fundamental concepts of Lambda and Lambda events on which later learning will be based. It is
important that you master these concepts yourself.
Since you are mastering fundamental skills, you are permitted to work from the examples in the
MySCU site or textbook, but you must acknowledge assistance from other textbooks or classmates. In
particular, you must not use online material or help from others, as this would prevent you from
mastering these concepts.
This diagram will help you understand where you can get help:
Encouraged Attribution Required Not acceptable Ask tutor
Be aware that if you do get help from one of the red sources, you will be reported for academic
misconduct, which may have serious penalties. Please visit the following link for the guidelines:
https:
it.ly/scuAcadMisconduct
Lecturer Tutors
Online
Forums
Relatives
Students
outside unit
Hired coders Classmates
Private
Tutors
Othe
https:
it.ly/scuAcadMisconduct
PROG2003 – Cloud Systems Development
Assignment 2
ATTENTION: Assignment 2 must be completed in the UA-provided AWS account. Personal AWS
accounts will not be accepted, and there will be a significant mark deduction. If your AWS account
is not accessible for any reason and the marker cannot check your app, you will also lose significant
marks. No excuse will be considered. You must use Java and relevant AWS SDK to implement your
app.
Details: The details of the assignment tasks are given below.
Part A: (30 marks):
• Create an S3 bucket named “yourscuusername-a2-bucket”.
• Implement a Lambda function named “yourscuusername-a2-function”. Add a “create object”
trigger for the bucket “yourscuusername-a2-bucket” to this Lambda function.
• Use the “LabRole” as the execution role of the Lambda function. Add an inline policy to this
ole so that “LabRole” allows DynamoDB actions for all resources. If you find that an inline
policy is already added to your “LabRole” for DynamoDB access and it works, then you can use
that; no need to create a new one.
• The Lambda function is invoked only when a new object named “Student.json” is
created/uploaded in “yourscuusername-a2-bucket”. The “Student.json” includes a JSON
a
ay with more than one JSON objects. An example of the JSON a
ay is given below. A sample
Student.json is attached with this assignment. You can add new elements to this JSON file.
[{"Name": "Jacob", "Age": 21, "Degree": "Bachelor of IT", "Credits": 24},
{"Name": "Sarah", "Age": 18, "Degree": "Bachelor of IT", "Credits": 30},
{"Name": "Marcus", "Age": 24, "Degree": "Bachelor of IT", "Credits": 36},
{"Name": "Peter", "Age": 29, "Degree": "Master of IT", "Credits": 40},
{"Name": "Tori", "Age": 19, "Degree": "Master of IT", "Credits": 12}]
• Create a Cloud9 app named “yourscuusernameA2App”. Implement a handler named
“handleRequest()” with appropriate parameters to receive S3 event notifications.
• The handler retrieves the bucket name and object key associated with the S3 event and
conducts the following tasks only for the “Student.json” object. For any other object
created/uploaded the handler will not do anything following.
• The handler reads the JSON a
ay in the “Student.json”, parses the JSON objects, and inserts
the values of the fields to a DynamoDB table named “Student”. The schema of the table is
given below, where Id is the primary key. You can add a test item to the table as shown below.
Student(Id: Number, Name: String, Age: Number, Degree: String, Credits: Number)
• The handler must retrieve the latest value of “Id” from the DynamoDB table before doing any
insertion.
• You must use AWS SDK v1.x, appropriate exception handlings, loops, and separate methods
for insertion and reading the latest Id.
• Add detailed comments to your Java source code. JSON parsing must be done by “org.json”
package.
PROG2003 – Cloud Systems Development
Assignment 2
Testing the Lambda function: You can test the Lambda function by just simply uploading the
Student.json into the specified bucket.
Solution Helps: Check the following hints for implementing Part A.
• You need to use the “org.json” package to retrieve the JSON a
ay and parse each JSON object.
Please check this link: https:
stleary.github.io/JSON-java/index.html. You can use the latest
maven dependency from here: https:
mvnrepository.com/artifact/org.json/json.
• A JSONA
ay is a collection of JSONObjects and a JSONObject is a collection of values of each
data filed. You need to understand this concept and parse the fields.
Part B: (10 marks):
• You need to create, publish, and import a recipe named “yourscuusername-recipe” to
complete the following tasks on the “resolution.csv” (attached with this assignment) in the
Glue DataBrew project. Create a project named “yourscuusernameA2Project”.
• Replace all missing values in any two columns which have missing values. Use the most
frequent value and numeric aggregate to compute the missing value.
• Identify the duplicate rows and flag them with “yes or no” values.
• Flag outliers for any one column as “yes or no” values.
• Compute the total number of “no votes” for each distinct “vote date” and add a new column
to the dataset.
• Compute the total number of “yes votes” for each distinct “vote date” and add a new column
to the dataset.
• Use “LabRole” while creating the project. Upload the “resolution.csv” to “yourscuusername-
a2-bucket”.
• Import the recipe as a JSON file.
Solution Helps: No solution hint is required for implementing Part B.
Submission Checklist: The marker will access your AWS Academy workspace (provided by the UA),
check your app, and directly mark your app from there. You have to zip the app project folder and
submit it in Blackboard as well. Blackboard submission list:
• Zipped app folder for the Handler app.
• The recipe JSON file.
https:
stleary.github.io/JSON-java/index.html
https:
mvnrepository.com/artifact/org.json/json