Hi Joe, Yahoo Pipes turned on a cap for their service, so if more than 200 requests are received from the same IP address within an hour, it will start returning a 999 error for that pipe for the next hour.

For some reason, our instances of feed manager on the library site seem to be tripping this, although we have the cache time set to 15 minutes. I asked our library's webmaster to up the cache time to 60 minutes, remove our local RSS feed for the feed manager, and take down the feeds until Yahoo clears the 999 error for the pipe, so we can try again.

The problem I thought should perhaps be addressed or worked around is that in looking at the system log, it seems once an RSS error is returned to feed manager, it keeps attempting to load that RSS feed on every page load by every user. Of course, that means that this particular error will never go away, because between bots, etc., we're definitely getting more than 200 hits per hour on the home page of the site. 

Would it be difficult to change feed manager so that it respects the cache time, whether an error was returned or not? Maybe this should be done via a system key, since some people may want it to constantly poll if an error occurs.

Hi Joe, there is only one feed in the feed manager instance, so that's most likely the problem.

Actually, the only reason we're using Pipes is because the original order of the underlying RSS feed is getting scrambled by feed manager, since there's no pubDate included on it. I used Pipes to process the description field, and strip out the date to create a pubDate field in the reprocessed RSS.

If you can work out a way to have feed manager display the items in the same order as the original feed delivers them, we wouldn't need pipes at all (and that would be the best solution of all). Here's the URL of the RSS feed we're reprocessing, so you can play with it yourself and see if that's a possibility. It does pass the W3C validation, so I guess pubDate is an optional field.

http://host7.evanced.info/escondido/lib/eventsxml.asp?ag=&et=&lib=ALL&nd=30&dm=rss2&LangType=0

Thanks Joe!

Jamie

Thanks Joe, I think that's where we left it last time. So the problem is that the feed manager assigns a pubDate to the entries with precision only to the second, so most (probably all) of the entries have the same pubDate.

If I do the code changes to implement, would you be okay with having a new recv_seq field on the mp_rssfeedentries table, along with the code to sort the output first by pubDate, then recv_seq? That seems like it would be an easier and safer change than trying to switch the date fields to TIMESTAMP(6) with fractional seconds, especially since support for storing fractional seconds was only added to MySQL recently (version 5.6.4).

If it's a go, it may be a little while until I can do this as I'm going on vacation for a couple of weeks after today.

Jamie