Hello,
A bit of a backstory with the webhook:
We're using an account webhook (we didn't use the folder one since it doesn't trigger for subfolders) to keep files in sync with a local zip that we keep.
The hook worked perfectly until i'd say 2 weeks ago when it sometimes would not trigger at all, or we'd get the events with either a few minutes delay, or the next day.
We know that some didn't trigger since we keep both a db log and a file log of all hooks (we log the event before we even begin to process it).
Is there a place to see all queued/failed events? And if not, can we provide some account/hook/folder ids to be checked?
Thanks,
Andrei
A bit of a backstory with the webhook:
We're using an account webhook (we didn't use the folder one since it doesn't trigger for subfolders) to keep files in sync with a local zip that we keep.
The hook worked perfectly until i'd say 2 weeks ago when it sometimes would not trigger at all, or we'd get the events with either a few minutes delay, or the next day.
We know that some didn't trigger since we keep both a db log and a file log of all hooks (we log the event before we even begin to process it).
Is there a place to see all queued/failed events? And if not, can we provide some account/hook/folder ids to be checked?
Thanks,
Andrei
Posted 1 year ago
Dale Smith, Software Engineer
Dale Smith, Software Engineer
Robert Jakubowski, Software Engineer
Robert Jakubowski, Software Engineer
Robert Jakubowski, Software Engineer
Sirbu, Lukasz
The fix has been deployed. Let us know if you still experience any issues.
Thanks,
Robert
The fix has been deployed. Let us know if you still experience any issues.
Thanks,
Robert
Thanks.
Wii do.
Wii do.
Robert Jakubowski, Software Engineer
Thanks for bringing it up.
We have identified an intermittent issue that occurred end of last week which should be resolved now. Sorry for that.
Are you still experiencing the problem?
We have identified an intermittent issue that occurred end of last week which should be resolved now. Sorry for that.
Are you still experiencing the problem?
Robert - this is still happening - I just checked it and I am not receiving webhook events when i drop a file.
Any update on this Robert? I tried openning a supprt ticket, but they closed it within acouple of hours with no resolution... I don't know what I should do at this point and it's affecting our live systems.
@Robert - following up on this once again.
Robert Jakubowski, Software Engineer
@Łukasz, could you please provide the support ticket number?
I will follow up with our support channel. Unfortunately I am not able to help more at the moment.
And sorry again for the inconvenience.
I will follow up with our support channel. Unfortunately I am not able to help more at the moment.
And sorry again for the inconvenience.
@Robert - case number is 78819328 - it was closed within a couple hours with no resolution on your end.
Robert Jakubowski, Software Engineer
Thanks. I have requested internally to have the case revisited.
Hey Robert - it was not reopened. Could you please look into this?
Robert Jakubowski, Software Engineer
Hi Łukasz,
I have raised your issue several times but my request is still pending.
I will add a note that you just asked.
My apologies again.
I have raised your issue several times but my request is still pending.
I will add a note that you just asked.
My apologies again.
I will be monitoring this this week and let you know, however i still believe i was not receiving all of the events Today. I will provide an update tmrw.
Matthew, Product Manager
Hi Łukasz,
We are unable to reopen your prior support case, however, we will be investigating the root cause of your webhooks issue. I will provide an update when we have identified the cause and when a fix is in place.
We are unable to reopen your prior support case, however, we will be investigating the root cause of your webhooks issue. I will provide an update when we have identified the cause and when a fix is in place.
(Edited)
Matthew,
Any update on this?
Any update on this?
Anuja Gogate, Employee
Hey Lukasz,
Could you provide us with the following information:
Webhook Subscription Id (this is the id generated when you created the webhook)
Your OAuth Client Id
The Url the webhook is going to
The specific event details that was late (for instance if it was File Upload event, the file name and folder id the event happened in)
A time frame that this was happening.
You can upload these details in a text file via: https://citrix.sharefile.com/r-r4089026e08b4b199
Could you provide us with the following information:
Webhook Subscription Id (this is the id generated when you created the webhook)
Your OAuth Client Id
The Url the webhook is going to
The specific event details that was late (for instance if it was File Upload event, the file name and folder id the event happened in)
A time frame that this was happening.
You can upload these details in a text file via: https://citrix.sharefile.com/r-r4089026e08b4b199
It looks like it's working now, even tough I did not change anything. Will monitor it again.
@Anuja - it was working correctly up until i activated a second webhook. That caused the same issue to reappear, which means that im only getting some of the webhook events back to my server. I added a logging to it, so i know that the requests do not even hit my servers.
Hello there,
Any reports about webhooks issues since last Thursday? It was working fine until then, but now even what we call the Webhooks subscription endpoint (https://companyname.sf-api.com/sf/v3/WebhookSubscriptions) we got the follow response:
{
"code": "NotFound",
"message": {
"lang": "en-US",
"value": ""
},
"reason": "NotFound"
}
Same response when we do POST, when trying to recreate subscriptions.
Thanks,
Any reports about webhooks issues since last Thursday? It was working fine until then, but now even what we call the Webhooks subscription endpoint (https://companyname.sf-api.com/sf/v3/WebhookSubscriptions) we got the follow response:
{
"code": "NotFound",
"message": {
"lang": "en-US",
"value": ""
},
"reason": "NotFound"
}
Same response when we do POST, when trying to recreate subscriptions.
Thanks,
Anuja Gogate, Employee
Claudio and I have exchanged communication on another thread and this issue is resolved.
Any updates here? We are experiencing this issue since Thursday 4/18 such that we are no longer receiving a webhook for every upload on an account-wide webhook subscription. This is very problematic because action based on these uploads is SLA bound.
Thanks,
Adam
Thanks,
Adam
Matthew, Product Manager
Thank you for sharing issues you are experiencing with our
service. We are unable to record customer issues and communicate the progress
of fixes on this forum. Please contact support to open a case here: https://www.sharefile.com/support.
Support will need to securely capture your account information to troubleshoot
the issue.
Anuja Gogate, Employee
Adam,
Like Matt mentioned, if possible please do open a support case. In the meantime, could you provide us with the following information:
Webhook Subscription Id (this is the id generated when you created the webhook)
Your OAuth Client Id
The Url the webhook is going to
The specific event details that was late (for instance if it was File Upload event, the file name and folder id the event happened in)
A time frame that this was happening.
You can upload these details in a text file via: https://citrix.sharefile.com/r-r4089026e08b4b199
Like Matt mentioned, if possible please do open a support case. In the meantime, could you provide us with the following information:
Webhook Subscription Id (this is the id generated when you created the webhook)
Your OAuth Client Id
The Url the webhook is going to
The specific event details that was late (for instance if it was File Upload event, the file name and folder id the event happened in)
A time frame that this was happening.
You can upload these details in a text file via: https://citrix.sharefile.com/r-r4089026e08b4b199
Matthew, Anuja -
I have already submitted a support case (78882083). Support was unable to assist, stating they "do not support" anything related to API nor webhooks. They directed me to post here. I would strongly suggest Citrix figure out how to support this product going forward. Terrible customer experience.
Anuja - Submitted information as zip. Events are not simply late: we are not receiving events at all for a significant number of file uploads.
Thanks,
Adam
I have already submitted a support case (78882083). Support was unable to assist, stating they "do not support" anything related to API nor webhooks. They directed me to post here. I would strongly suggest Citrix figure out how to support this product going forward. Terrible customer experience.
Anuja - Submitted information as zip. Events are not simply late: we are not receiving events at all for a significant number of file uploads.
Thanks,
Adam
Adam,
I had exactly the same issue like you, but it magically started working yesterday for every upload. Before that it would work for exactly every second upload for an account wide subscription.
Also like Adam noted I had the same issue with support, went with them over email back and forth for ~4 weeks just to have an answer that they do not support this. It was really a terrible experience.
I had exactly the same issue like you, but it magically started working yesterday for every upload. Before that it would work for exactly every second upload for an account wide subscription.
Also like Adam noted I had the same issue with support, went with them over email back and forth for ~4 weeks just to have an answer that they do not support this. It was really a terrible experience.
Sagar Thirumala, Software Engineer
Adam,
Thanks for submitting the details. Based on the investigation, we do see few calls to the URL you have provided succeeding. But for the failure cases, the URL doesn't seem to be responding back with in 10 seconds.
As per the documentation:
Thanks for submitting the details. Based on the investigation, we do see few calls to the URL you have provided succeeding. But for the failure cases, the URL doesn't seem to be responding back with in 10 seconds.
As per the documentation:
The ShareFile Platform expects a 2XX response back in 10 seconds to indicate a success. If we do not receive a successful response within that timeframe, we will consider it a failure.In general, consumers are recommended to accept callbacks quickly and process them asynchronously, hope this information helps for further troubleshooting on your end. Let us know if you need any more information.
Sagar, Anuja,
The issue is not on the receiving end, as nothing has changed there, and a response is provided immediately. The implementation conforms to your recommendation.
To be sure, we cleaned up our Webhook Subscriptions list and re-subscribed to the route in question using another URL that would hit the same destination on Friday 4/26 @ ~5:30 PM Pacific Time. That same URL was verified independently as working via Postman tests. Since Friday we are not receiving any inbound webhooks. This happened a few months ago in the same fashion, and the issue was squarely on your end.
What is going on here? We need a fix fast.
Thank you,
Adam
The issue is not on the receiving end, as nothing has changed there, and a response is provided immediately. The implementation conforms to your recommendation.
To be sure, we cleaned up our Webhook Subscriptions list and re-subscribed to the route in question using another URL that would hit the same destination on Friday 4/26 @ ~5:30 PM Pacific Time. That same URL was verified independently as working via Postman tests. Since Friday we are not receiving any inbound webhooks. This happened a few months ago in the same fashion, and the issue was squarely on your end.
What is going on here? We need a fix fast.
Thank you,
Adam
Update? Still no webhooks being received.
Sagar Thirumala, Software Engineer
Adam,
My apologies for the delay in response, we should have the fix available since last couple days, the fix will ensure that we retry calling the registered url with exponential back-off strategy for timeout scenarios.
Let us know if you notice any further issue with not receiving webhook calls.
My apologies for the delay in response, we should have the fix available since last couple days, the fix will ensure that we retry calling the registered url with exponential back-off strategy for timeout scenarios.
Let us know if you notice any further issue with not receiving webhook calls.
Sagar,
We have uncovered more Webhook misses from the past few days, with a significant number yesterday evening. I just uploaded examples to the ShareFile upload drop previously provided + details requested about the webhook subscription.
What's going on there? This is how the issues began previously.
Thank you,
Adam
We have uncovered more Webhook misses from the past few days, with a significant number yesterday evening. I just uploaded examples to the ShareFile upload drop previously provided + details requested about the webhook subscription.
What's going on there? This is how the issues began previously.
Thank you,
Adam
Sagar Thirumala, Software Engineer
Adam,
My apologies for the delay in response, could you confirm if you guys saw any further upload drops since 11th May?
My apologies for the delay in response, could you confirm if you guys saw any further upload drops since 11th May?
Leo, Official Rep
