Create a client library for an external REST Service
When developing backend applications, in order to provide a functionality, we may need to interact with other REST services. Most popular services may have client libraries in different languages like C#, JS, Python. However, there are situations where a service provider does not offer such library for the language that we are currently working with. What to do in such case? Can't we just do some HTTP request directly in our business logic? What about authentication, how to handle expirable tokens? These and more questions will be answered in this article.
First things first
For the sake of this article, I picked Cat API which as stated in the official page offers an API for retrieving cat information such breeds, categories etc.
A public service API all about Cats, free to use when making your fancy new App, Website or Service.
Cat API does not offer a client library for consuming the service in Go. So we are going to create it for Go.
Creating the client
GoLang is more of a functional language but it also supports development in OOP style to some degree by offering interfaces and structs. In our case we are going to use a struct for the client which will hold all information needed by the client to operate.
type CatApiClient struct {
host url.URL
accessKey string
version string
httpClient *http.Client
}- host: it is the URL of the service and in our case is: https://api.thecatapi.com.
- accessKey: you can get one by signing up at Cat API.
- version: it is the api version such as v1, v2 etc.
- httpClient: it is the http client which carries out http requests for the client and is reused throughout the lifecycle of the client for efficiency.
In Go specification there are no constructors and for struct initialization we use functions. To create and initialize a CatApiClient, the function constructor is declared as following:
func NewClient(host, accessKey, v string) (*CatApiClient, error) {
u, err := url.Parse(host)
if err != nil {
return nil, ErrInvalidHost
}
apiClient := &CatApiClient{host: *u, accessKey: accessKey, version: v}
// create default http client which will execute requests.
apiClient.httpClient = &http.Client{Timeout: 10 * time.Second}
return apiClient, nil
}Before continuing, we need struct entities where response serialized data will be deserialized into. Lets take for example /breeds API which returns a list of breeds. In order to get the response format we make a request in Postman at breeds API and we get a response like this:
[
{
"weight": {
"imperial": "7 - 10",
"metric": "3 - 5"
},
"id": "abys",
"name": "Abyssinian",
"cfa_url": "http://cfa.org/Breeds/BreedsAB/Abyssinian.aspx",
"vetstreet_url": "http://www.vetstreet.com/cats/abyssinian",
"vcahospitals_url": "https://vcahospitals.com/know-your-pet/cat-breeds/abyssinian",
"temperament": "Active, Energetic, Independent, Intelligent, Gentle",
"origin": "Egypt",
"country_codes": "EG",
"country_code": "EG",
"description": "The Abyssinian is easy to care for, and a joy to have in your home. They’re affectionate cats and love both people and other animals.",
"life_span": "14 - 15",
"indoor": 0,
"lap": 1,
"alt_names": "",
"adaptability": 5,
"affection_level": 5,
"child_friendly": 3,
"dog_friendly": 4,
"energy_level": 5,
"grooming": 1,
"health_issues": 2,
"intelligence": 5,
"shedding_level": 2,
"social_needs": 5,
"stranger_friendly": 5,
"vocalisation": 1,
"experimental": 0,
"hairless": 0,
"natural": 1,
"rare": 0,
"rex": 0,
"suppressed_tail": 0,
"short_legs": 0,
"wikipedia_url": "https://en.wikipedia.org/wiki/Abyssinian_(cat)",
"hypoallergenic": 0,
"reference_image_id": "0XYvRd7oD",
"image": {
"id": "0XYvRd7oD",
"width": 1204,
"height": 1445,
"url": "https://cdn2.thecatapi.com/images/0XYvRd7oD.jpg"
}
},
...
]We need to convert the object inside the JSON array to a Go struct. Luckily for as there is an online tool where we can create the required Go struct for that JSON Object.
We change struct name from AutoGenerated to Breed and save it in a file breed.go. You may also extract abstract struct such Image property and store it in a separate struct like ImageDescriptor(this is not required)
Now that we have the required data models for the breed API we can continue to the next topic.
Create client helper methods
For maximum code reusability, it is necessary to see the big picture and think about it before going straight into writing a certain functionality. With that I mean that we need to think about authentication and request construction which both can be reused between different API calls. This increases code reusability and it is easier to maintain. A total disaster would be if such logic is scattered among different API and when we find a bug in one API we would need to review and fix all API-s where such logic is copy-pasted. So its is a maintenance nightmare.
For the Cat API I created a generic purpose method for performing a GET request. It takes three parameters as shown in the following code snippet:
- path: defines the path of the request without including host or version; both host and version are handled automatically inside this function.
- query: it is basically a map with query parameters that will be appended to the request
- output: it is the structure where the response will we deserialized into; for example for breed list we pass a pointer to a Breed slice.
func (c *CatApiClient) execGet(path string, query url.Values, output interface{}) error {
url := c.host
url.Path = "/" + c.version + path
fmt.Println(url.String())
req, err := http.NewRequest(http.MethodGet, url.String(), nil)
if err != nil {
return err
}
req.Header.Add("x-api-key", c.accessKey)
req.URL.RawQuery = query.Encode()
resp, err := c.httpClient.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode != 200 {
errBytes, _ := io.ReadAll(resp.Body)
return errors.New(string(errBytes))
}
dec := json.NewDecoder(resp.Body)
return dec.Decode(output)
}The method is also responsible for taking care of authentication. As it can be seen in the code, in the line 12 we add the x-api-key header to the request.
Get Breeds API
Finally all required logic is in place so we can with not much effort complete the first Cat API call. Making use of execGet too much logic is abstracted away and the GetBreeds method is soo thin as shown in the following snippet:
func (c *CatApiClient) GetBreeds(query url.Values) ([]*Breed, error) {
var breeds []*Breed
if err := c.execGet("/breeds", query, &breeds); err != nil {
return nil, err
}
return breeds, nil
}Basically we have 9 loc for one API. WHAAAA!
Search Breeds API
Let's extend the Cat API Client with a search functionality for demo purposes. The response format does not change so we have the Breed struct and all required data models in place.
func (c *CatApiClient) SearchBreeds(query url.Values) ([]*Breed, error) {
var breeds []*Breed
if err := c.execGet("/breeds/search", query, &breeds); err != nil {
return nil, err
}
return breeds, nil
}As expected also this API call did not forced us to write any extra logic for making a request to Cat API.
Lazy Authentication
There was a topic that I really wanted to discuss there about a specific scenario that I encountered at my work. What if our access key is expirale? In such case we would need to renew it. The best is to handle this process as smoothly as possible. Unfortunately, Cat API has a non expirable access key so the code from now on will be hypothetical just for demonstration purposes.
First of all we need a new method for handling refresh token/access key procedure. This method will be called in low level helper functions such as execGet for better code reusability.
func (c *CatApiClient) execGet(path string, query url.Values, output interface{}) error {
if c.hasTokenExpired() {
if err := c.refreshToken(); err != nil {
return err
}
}
url := c.host
url.Path = "/" + c.version + path
...
}
func (c *CatApiClient) refreshToken() error {
c.refreshMutex.Lock()
if c.hasTokenExpired() == false {
return nil
}
accessKey, err := retrieveNewToken("token refresh string or credentials")
if err != nil {
return err
}
c.accessKey = accessKey
expiry := 120 * time.Second // define token expiry in seconds or whatever and add offset to avoid invalid expiry
c.tokenExp = time.Now().Add(expiry)
c.refreshMutex.Unlock()
return nil
}
func (c *CatApiClient) hasTokenExpired() bool {
t := time.Now()
return t.After(c.tokenExp)
}The refresh token procedure starts with a mutex lock. You may ask why we need one? In such case I did not want to call the retrieveToken multiple times to get different tokens because that would be just a waste of resources. Instead only one caller who gets the lock can enter inside the critical region(code between lock and unlock).
After the lock is passed, a check for token expiration is done. If the token has expired, a new one is retrieved calling retrieveNewToken. For other callers which are blocked by the mutex, this check condition will be satisfied as the token is renewed by the first call.
After the token is retrieved, client struct properties will be updated accordingly. Here it is important to mention the calculation of the expiry for the token. I would suggest to set an expiry earlier that the amount given in the service authentication specification by 5 - 10s so we avoid a false positive in token expiry check.
Conclusions
Sometimes we may need to interact with external REST API services which may not have a client library in the language our application is being coded. In such case the best approach is to abstract the interaction with the said service in a dedicated entity for better testing and maintenance. It is critical to provide a smooth authentication mechanism which supports token token renewal under the hood. A good attention should be paid to the code reusability which is an investment in the long-term. It is necessary to see the big picture and think about it before going straight into writing a certain functionality.
The source code can be found there.