Lecture Presentation

Goals

  • Perform a network request fetching data over http
  • Why and how to perform asynchronous operations in Java
  • Using public web APIs and interpret data in JSON format

Threading

  • General term for running multiple code blocks at the same time
    • Or at least switching between them fast enough for humans not to see the difference
  • Multithreaded code
    • Must be intelligent enough to realize that the same code block can run at the same time in multiple instances
  • User interface thread
    • In Android, as in most operating systems, all user interface code runs in a single thread and must do so

Dangers of slow code

  • Code running in the user interface thread should always finish as soon as possible
  • Any operation that could take a while to complete should be done in a separate thread
  • A while might be a second or even less
  • Android will warn user about application might be dead
    • If an activity event handler takes more than 5 seconds
    • If a service handler takes more than 10 seconds

Difficulties with running code in a separate thread

  • Access to any shared data must be serialized
    • Can often be avoided by using intelligent constructs in the Android platform
  • Views are not allowed to be accessed in any way, can't call any View methods
    • Any user interface updating must be specified to run on the user interface thread
  • Activity might die for any reason, like memory low or user switching to other app
    • Code in separate thread should not hang around after its associated Activity has died

AsyncTask class

  • Great helper class for doing things in the background
  • Takes care of
    • Running the main task on a background thread
    • Switching to the main ui thread to do post processing (and pre)
    • Optional switching to the main ui thread during the background processing
      • For any progress bar updating or similar than should be updated. Requires that you know how far along you are when doing the background processing.
  • Generic class that takes three types
    • Type that tells the task what to do
    • Type that tells the progress updater how far along the task is
    • Type that contains the result from the task when doing the post processing

AsyncTask overridable methods

  • Extend your own class from AsyncTask
  • Override methods for the events that you want to handle
    • onPreExecute
      • Often not used since most user interface updating before the operation starts can just as easily be done before running the AsyncTask. Is run on main UI thread.
    • doBackground
      • This is where the long running code can be performed. Is run on a separate thread. Can't access anything in the user interface. Returns the value that is the argument to onPostExecute.
    • onProgress
      • This can be called indirectly from doBackground by calling updateProgress. Is run on main UI thread.
    • onPostExecute
      • Called with the returned value from doBackground. Is run on the main UI thread so the user interface can safely be updated here.
  • Create object from class
    • And then call execute to start the task

Example

  • AsyncTask
    AsyncTask task = new AsyncTask<String, Void, String>() {
      @Override
      String doBackground(String... params) {
        // params[0] is the first argument to execute
        return "";
      }
      @Override
      void onPostExecute(String result) {
        // result is whatever is returned from doBackground
      }
    };
    task.execute("Sweden");
    

Network requests

  • Android has two libraries for doing network requests
  • org.apache.http
    • Open source library not part of core Java
    • Included in all Android platform versions
    • Was the recommended way before Android version 3.0
    • Classes like DefaultHttpClient, HttpGet, HttpPost, HttpResponse
  • java.net
    • Included in all Java platforms, including all Android platform versions
    • Is now the recommended way from Android version 3.0 and going forward
    • Classes like URL, URLConnection, HttpURLConnection
  • Basic principles are the same, but the classes and methods are quite different
  • We'll focus on java.net in this course

Network permission

  • An application must request network permission
  • AndroidManifest.xml
    <uses-permission android:name="android.permission.INTERNET" />
    

Preparing a network request

  • We're assuming all network requests will use http
  • Setup the address
    URL address = new URL("http://www.cnn.com");
    URLConnection conn = address.openConnection();
    
  • If you need to set http specific information
    URL address = new URL("http://www.cnn.com");
    HttpURLConnection conn = (HttpURLConnection)address.openConnection();
    conn.setRequestProperty("User-Agent", "Mozilla/5.0");
    
  • Note that the server is not contacted until a method that requires the response is called by you

Reading the response from the server

  • We need to find the correct encoding for non-us letters
    String encoding = conn.getContentEncoding();
    if (encoding==null)
      encoding = "utf-8";
    
  • Use a BufferedReader to read line-by-line into a StringBuffer
    BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream(),
      Charset.forName(encoding)));
    String line;
    StringBuffer all = new StringBuffer();
    while ((line = in.readLine())!=null) {
        all.append(line);
    }
    in.close();
    String html = all.toString();
    

Determine connectivity

  • No need to perform network requests if the network is down
  • Network requests can often fail intermittently
    • Be prepared to retry again before showing any error messages to the user
  • Check if there is a network available
    ConnectivityManager man = (ConnectivityManager)getSystemService(Context.CONNECTIVITY_SERVICE);
    boolean isOnline = man.getActiveNetworkInfo().isConnectedOrConnecting();
    
  • Needs specific permisson in AndroidManifest.xml
    <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
    

Web APIs

  • APIs
    • A collection of methods to perform actions and/or retrieve data in a structured manor suitable for programmed applications
  • Web APIs
    • An API that is available to call over http (and/or https)
  • Public Web APIs
    • Not a common title perhaps, but my way of entitling a web API that can be called without any registration, keys, authentication or similar a public API.
  • Find a Web API

Web/Screen scraping

  • When there is no API available, or for some other reason it is not usable
  • Read the html from the web site as any web site visitor using a browser
  • Identify, collect and extract the relevant information
  • Advantages
    • Very hard to block access (even though some try hard with encrypted/convoluted JavaScript etc)
    • Not dependant on the web application having an API
  • Disadvantages
    • Extremely sensitive to changes in the html
    • Will need to update code when the html is changed by the web site publishers
  • The Semantic Web
    • A movement where web sites that does not have an API, still tries their best to tag and enclose the html in such a manor that it is easy to identify, collect and extract relevant information

Interpret html using Java string handling

  • Find a string within a string
    int find = html.indexOf("id=\"main\"");
    
  • Find the next occurence of a string within a string
    int find = html.indexOf("<td");
    int secondFind = html.indexOf("<td", find);
    int thirdFind = html.indexOf("<td", secondFind);
    
  • Extract part of a string
    int find = html.indexOf("<td");
    int findStart = html.indexOf(">", find);
    int findEnd = html.indexOf("</td>", findStart);
    String part = html.substring(findStart+1, findEnd);
    

Interpret html using regular expressions in Java

  • Regular expressions are a large topic in itself for searching in text
  • Match contents of an element with specific id
    Pattern exp = Pattern.compile("<td.+?id=\"headline\".+?>(.*?)</td>");
    Matcher match = exp.matcher(html);
    if (match.find()) {
      String part = match.group(1);
    }
    
  • Explained
    • . means any character
    • 1. means one or more occurences of previous character
    • * means zero, one or more occurences of previous character
    • ? after * or + means to stop matching as soon as possible
    • ( and ) means to match a group that we can extract later
    • All other characters are literal characters matched to the html source

Interpret JSON

  • Many public web APIs return JSON formatted data
  • Android platform includes org.json library for working with JSON
  • Interpret a JSON string and extract arrays, objects and properties
    // Assume json string contains:
    // { title: "Name", items: [ 1, 2, 3 ] }
    JSONObject obj = new JSONObject(json);
    String title = obj.getString("title");
    JSONArray items = obj.getJSONArray("items");
    int secondValue = items.getInt(1);
    

Binary data from a network request

  • Handle downloaded binary data
    • If the response from the server is binary, like an image or an mp3 file, you probably want to store it on the local file system.
  • Most code similar to fetching html over the network
    • Get a suitable directory on the local file system where to store the file
    • Open a file in that directory and write to it while reading the network response
  • Example
    File outputFile = new File(getExternalCacheDir().getAbsolutePath() + "/myfile.jpg");
    URL url = new URL("http://anysite.com/images/test.jpg");
    URLConnection conn = url.openConnection();
    byte[] buffer = new byte[4096];
    int readCount;
    InputStream input = conn.getInputStream();
    FileOutputStream output = new FileOutputStream(outputFile);
    while ((readCount = input.read(buffer))>=0) {
      output.write(buffer, 0, readCount);
    }
    input.close();
    output.close();
    

Some API examples

API Libraries / SDKs

More information

  • Book references
    • Android Programming Unleashed, Chapter 13 Creating and Consuming Services, Accessing data from the Internet
    • Android Programming Unleashed, Chapter 13 Creating and Consuming Services, Using the AsyncTask Class
    • Android Cookbook, Chapter 13.2 Using a RESTful Web Service