External rest data source authentication issue

  • 3
  • Question
  • Updated 3 years ago
  • Answered
I am having trouble connecting to a rest data source. It seems to be connecting to the right end point, but the only field it returns is the "message" field and when I add the message field to a table, the message field populates with "Authorization has been denied for this request." I have checked my user name and password several times. I have tried different settings for the data connection, but they all result in error messages that the data resource could not be found. Any Ideas?
Photo of Skuidward Tentacles (Raymond)

Skuidward Tentacles (Raymond), Champion

  • 17,224 Points 10k badge 2x thumb

Posted 3 years ago

  • 3
Photo of JD Bell

JD Bell, Senior Product Engineer

  • 2,996 Points 2k badge 2x thumb
Do you know what the expected authentication/authorization flow of your REST service is? And how credentials/tokens are supposed to be passed to the service? If so, please let us know so that we can advise you on how to accomplish that with Skuid.

===

More depth:

REST service authentication/authorization can be very complicated.

First, you need to determine how the REST service handles authentication and authorization.

If the REST service uses OAuth (or something like it), then you will need to communicate with an authentication service to exchange your authentication credentials (e.g., username and password) for an authentication token. You'd then exchange the authentication token with your resource service to get an access token (basically, a session Id). Only when you have the access token can you begin requesting data from the service.

Some REST services handle both authentication and authorization, but in separate steps. You'd call one function on the REST service to login, at which point you'd get an authorization or session token, then you'd use that token in lieu of credentials for your remaining requests.

Some REST services simply expect your credentials with every single request, and re-verify those credentials every time a function is called.

Once you've figured out your authentication/authorization flow, you still need to determine how the REST service expects to receive those credentials/tokens. Should they be included as URL parameters, in the request header, in the body? While some authentication flows have a specification (like OAuth and HTTP Basic Authentication), there's no guarantee that your service perfectly obeys those specs. For example, popular services like Facebook have authentication flows that closely resemble OAuth but are different enough to be incompatible.

Skuid was designed to be very flexible and work with a wide variety of authentication/authorization flows, so once you've determine the correct way to authenticate/authorize your user with the service, it should be possible to set that up in Skuid.
(Edited)
Photo of Skuidward Tentacles (Raymond)

Skuidward Tentacles (Raymond), Champion

  • 17,224 Points 10k badge 2x thumb
Thank you for the response. Your explanation helped me understand what is happening ( I think). Authentication seems to be working with "Shared User Name: Separate Authentication URL", but then I need to retrieve the token and pass it in the header of each call. I see the Headers to send with every request section, but I can't do anything with it. Under "Object" the only thing listed is "Blank Object". Any guidance on how I can accomplish the connection would be greatly appreciated. Thanks
Photo of Ben Hubbard

Ben Hubbard, Employee

  • 12,470 Points 10k badge 2x thumb
Hi Raymond,

The JSON Editor that we have right now is pretty unintuitive. You have to click on the "menu" icon to the left of the (empty object) label. Then you can click "Append". After that, you can put in the headers that you need. For the "value" part of the header, you will probably want to merge in a value from your previous authentication request. It will be something like this...


{{$Auth.Response.Body.myToken}}

or this...

{{$Auth.Response.Headers.myResponseHeader}}
(Edited)
Photo of Skuidward Tentacles (Raymond)

Skuidward Tentacles (Raymond), Champion

  • 17,224 Points 10k badge 2x thumb
Ok, thanks. I tried both of those header values and got no satisfaction. Is there a way to find what the merge variable would be for my particular case? My auth URL end point is /security/token if that helps....
Thanks
Photo of Ben Hubbard

Ben Hubbard, Employee

  • 12,470 Points 10k badge 2x thumb
You will need to inspect the payload and headers that come back from the request to /security/token. If you can post that here (scrub out any tokens or credentials), then I can probably tell you what your merge value needs to be. Also, it's important what your service is expecting in terms of the token. A lot of times it's a header called "Authorization" and then the value would be "Bearer {{$Auth.Response.Body.access_token}}. But this really varies from service to service. Is your service a public one that I could take a look at the documentation?
(Edited)
Photo of Skuidward Tentacles (Raymond)

Skuidward Tentacles (Raymond), Champion

  • 17,224 Points 10k badge 2x thumb
Thanks. Here is the API documentation: https://github.com/orionadvisor
Photo of Ben Hubbard

Ben Hubbard, Employee

  • 12,470 Points 10k badge 2x thumb
Hi Raymond,

I looked through the documentation, but couldn't find anything about the /security/token endpoint. Usually there's a section of the documentation that talks about how to authenticate.
Photo of Skuidward Tentacles (Raymond)

Skuidward Tentacles (Raymond), Champion

  • 17,224 Points 10k badge 2x thumb
Thanks Ben, these are the header requirements:

This goes in an 'Authorization' header:
'Basic '+EncodingUtil.base64Encode(Blob.valueOf(userName+':'+password))

Any help of how I would formulate that would be greatly appreciated.
Photo of Ben Hubbard

Ben Hubbard, Employee

  • 12,470 Points 10k badge 2x thumb
Oh, that's just basic http auth. We have that built in! Just use the basic http auth setting and enter your shared username and password. Per user credential storage is coming in an upcoming release.
Photo of Skuidward Tentacles (Raymond)

Skuidward Tentacles (Raymond), Champion

  • 17,224 Points 10k badge 2x thumb
Thanks Ben, but when I use the basic setting, then I go to the page I am trying to build, on page load of the page builder, I get this error message: . "Error retrieving metadata for Model(s) associated with Data Source 'OrionConnect'. Please check Models' properties to ensure they are set correctly. Error connecting to REST Data Source at URL "https://api.orionadvisor.com/api/v1/p...": Unauthorized". That is why I switched to the separate auth URL setting. When on this setting, I don't get the error message. I also don't get any data, so maybe it is the same thing....
Photo of Ben Hubbard

Ben Hubbard, Employee

  • 12,470 Points 10k badge 2x thumb
Do you know what version of Skuid you have installed?
Photo of Skuidward Tentacles (Raymond)

Skuidward Tentacles (Raymond), Champion

  • 17,224 Points 10k badge 2x thumb
7.27
Photo of Ben Hubbard

Ben Hubbard, Employee

  • 12,470 Points 10k badge 2x thumb
Hmm, that's strange. It seems like it should be working unless your username and password are somehow wrong.
Photo of Skuidward Tentacles (Raymond)

Skuidward Tentacles (Raymond), Champion

  • 17,224 Points 10k badge 2x thumb
Yes, strange. I have checked it many times. I can use the same credentials to log in directly to their application and into their API tester and both work fine.
Photo of Skuidward Tentacles (Raymond)

Skuidward Tentacles (Raymond), Champion

  • 17,224 Points 10k badge 2x thumb
Could you show me how I would create the basic header and I can try using the Separate Auth URL setting?
Photo of JD Bell

JD Bell, Senior Product Engineer

  • 2,996 Points 2k badge 2x thumb
Hey Raymond,

Based on the sample code in the GitHub repro, it looks like you must first connect to security/token using Basic authentication (putting the username and password in the Authorization header). The response you get back will be a JSON object with a property named access_token.

You would then include the access token in your API calls in the Authorization header, using something like "Session ".

To support this with Skuid, select the separate authentication URL option. (https://api.orionadvisor.com/api/v1/security/token)

In the headers to send with the auth request, add the Authorization header with a value of "Basic {{$Auth.BasicAuth}}".

Then, in your headers to send with every request, include an Authorization header with the value "Session {{$Auth.Response.Body.access_token}}".

If my reading of the sample code for this project is correct, then the above should work for you. But you'll need to check with the REST API developer/owner, since I could not find any documentation to back this up.
(Edited)
Photo of Skuidward Tentacles (Raymond)

Skuidward Tentacles (Raymond), Champion

  • 17,224 Points 10k badge 2x thumb
Thanks for the help on this. When setting up the headers, there is a "field" and a "value" . What should I be entering as "field"?
Photo of JD Bell

JD Bell, Senior Product Engineer

  • 2,996 Points 2k badge 2x thumb
Assuming I understand the REST service correctly:

Field = "Authorization", Value = "Session {{$Auth.Body.access_token}}"
Photo of Skuidward Tentacles (Raymond)

Skuidward Tentacles (Raymond), Champion

  • 17,224 Points 10k badge 2x thumb
That is what I assumed. No luck. I thank you for your help. I'll have to go to the developer. I doubt I will get their attention before January. I'll post back here if I get this resolved. One last question. Is there a way to see what the actual get request Skuid is sending looks like?
Happy Holidays!
Photo of Skuidward Tentacles (Raymond)

Skuidward Tentacles (Raymond), Champion

  • 17,224 Points 10k badge 2x thumb
Merry Christmas Eve,
The API developer got back to me with the following links for more info:
http://forum.riadevelopers.com/post/a...
http://forum.riadevelopers.com/post/o...

To authenticate,
Call [GET] https://testapi.orionadvisor.com/api/...
Include a Basic authorization header with the userid/pwd base 64 encoded:

[Header]
Authorization: Basic {uid}:{pwd}

returns an "auth_token"

The auth_token can then be used for all other api calls.

[Header]
Authorization: Session
Photo of Skuidward Tentacles (Raymond)

Skuidward Tentacles (Raymond), Champion

  • 17,224 Points 10k badge 2x thumb
HI. I'm still having issues with this. I just worked on it with my Salesforce Developer who has successfully build integrations with this API outside of Skuid. He reviewed your comments and my settings and says that everything is set correctly as far as we are able to tell. We can't actually see what is happening behind the scenes. The flow that JD Bell laid out in his comment is correct. Here is a screenshot of my settings. Still no satisfaction....
Photo of Matt Sones

Matt Sones, Champion

  • 31,478 Points 20k badge 2x thumb
Photo of Skuidward Tentacles (Raymond)

Skuidward Tentacles (Raymond), Champion

  • 17,224 Points 10k badge 2x thumb
Thanks Matt! I ran through this and the json is showing a post method instead of a get method. The tutorial, and my Salesforce developer indicate it should be a get. My model, however, is set to: Service Access Method = Get, so I'm not sure why it is using a post method
Photo of Skuidward Tentacles (Raymond)

Skuidward Tentacles (Raymond), Champion

  • 17,224 Points 10k badge 2x thumb
Unfortunately no satisfaction on this. I changed the verb to GET, but I still get this error when I load the page with the model based on the service in page builder:
1. Error retrieving metadata for Model(s) associated with Data Source 'OrionConnect'. Please check Models' properties to ensure they are set correctly. Error connecting to REST Data Source at URL "https://api.orionadvisor.com/api/v1/p...": Unauthorized
I checked my user ID and password several times and they are correct.
Photo of Zach McElrath

Zach McElrath, Employee

  • 48,984 Points 20k badge 2x thumb
Raymond, in your Common Request Headers, the following is invalid and will not return anything: {{$Auth.Body.access_token}}. Replace it with this:

Session {{$Auth.Response.Body.access_token}}
Photo of Skuidward Tentacles (Raymond)

Skuidward Tentacles (Raymond), Champion

  • 17,224 Points 10k badge 2x thumb
Ha! It works! It really works! You guys are awesome!
Now I have a couple follow up question.
1)It is bringing the data in, but it is in read only format. If I switch the model behavior to Read/Write the service URL and the service access method settings hide. If I then load the live page, it still brings in data, but is still read only. Is read/write possible?
2) I set up a model on an object that has 10's of thousands of records. It exceeded the governors and I got an error as expected, but unlike regular models, these models don't have regular conditions. They are URL merge conditions and when I try to create one it tells me, "This model's service URL does not contain any merge parameters". Also, there are no settings for max records or sort by. Is there currently any way for me to either cap the number of records queried or apply a condition to the model?
Photo of Zach McElrath

Zach McElrath, Employee

  • 48,984 Points 20k badge 2x thumb
(1) When you switch the Model to Read/Write, the Service URL and other properties get moved into a "Query"  Method within the Methods child node of the Model (next to Fields). If your REST API that you're using supports it, you can add additional Methods to provide the ability to Create, Update, and/or Delete records of the REST Resource in question. Whether this makes sense / is possible for your particular REST API depends on the API you're using.

(2) As far as limiting which / how many records are returned by a REST request, this is entirely dependent on the particular REST API you're using, in your case OrionConnect. Often, REST API's provide various URL Parameters that you can add to your REST Model's URL which will limit the number of records retrieved or apply the equivalent of "Conditions" to the request. For example with OrionConnect, the TradeFiles GET request lets you include startDate and endDate URI Parameters with your request to limit the range of TradeFiles requested. This is just a for instance --- I'm not sure which API you're using, there may not be any URI Parameters supported by the REST API you want to use, but typically if there's enough data being returned that Salesforce governor limits were exceeded, I'm betting that there's some URI Parameters you can use to restrict the payload size.
Photo of Skuidward Tentacles (Raymond)

Skuidward Tentacles (Raymond), Champion

  • 17,224 Points 10k badge 2x thumb
Great, thanks. I'll take a look.!
Photo of Skuidward Tentacles (Raymond)

Skuidward Tentacles (Raymond), Champion

  • 17,224 Points 10k badge 2x thumb
This reply was created from a merged topic originally titled External data issue.

Hello, I'm still seeking help with this issue. I think it may be a problem with Skuid, so I marked this post as "Problem". I am unable to connect to an external rest interface. Several community members tried to help and I ran a JSON diaginostic and found that my page is generating a post method instead of a get method. My Skuid page is set to method=Get, though. If anyone in Skuidland can help me out, I would greatly appreciate it. The details can be found in the below post:

https://community.skuidify.com/skuid/...

Thanks!