Descriptive Analysis using Spark SQL

ICS 574 – HW #2 Descriptive Analytics 1. Solve the following questions on Google Colab or Databricks using Spark SQL a. [4 pts] Search the internet for a big dataset of at least 0.5 GB. b. [4 pts] Create a DataFrame from the dataset. c. Using the DataFrame and implement the following aggregation functions. i. [4 pts] Aggregation with grouping ii. [4 pts] Aggregation with pivoting iii. [4 pts] Aggregation with rollups and cubes d. Spark SQL supports the following window functions. Apply these functions on the DataFrame i. [10 pts] Ranking functions 1. rank 2. dense_rank 3. percent_rank 4. row_number 5. ntile ii. [10 pts] Analytic functions 1. cume_dist 2. first_value 3. last_value 4. lag 5. lead Deliverables • One pdf file which contains the following. o A cover page which includes, your KFUPM ID, name, HW number, and date o A description of the big dataset and its source. o Each SQL statement and a snapshot of its output o Problems you faced if any. Note: • • Submit the homework before 11:59pm Saturday April 27, 2024. There are many YouTube videos that teach how to use Spark in Databricks or Colab.

Descriptive Analysis using Spark SQL

We offer the best custom writing paper services. We have answered this question before and we can also do it for you.


We offer the bestcustom writing paper services. We have done this question before, we can also do it for you.

Why Choose Us

  • 100% non-plagiarized Papers
  • 24/7 /365 Service Available
  • Affordable Prices
  • Any Paper, Urgency, and Subject
  • Will complete your papers in 6 hours
  • On-time Delivery
  • Money-back and Privacy guarantees
  • Unlimited Amendments upon request
  • Satisfaction guarantee

How it Works

  • Click on the “Place Order” tab at the top menu or “Order Now” icon at the bottom and a new page will appear with an order form to be filled.
  • Fill in your paper’s requirements in the "PAPER DETAILS" section.
  • Fill in your paper’s academic level, deadline, and the required number of pages from the drop-down menus.
  • Click “CREATE ACCOUNT & SIGN IN” to enter your registration details and get an account with us for record-keeping and then, click on “PROCEED TO CHECKOUT” at the bottom of the page.
  • From there, the payment sections will show, follow the guided payment process and your order will be available for our writing team to work on it.