an
/
go-pkg-rss
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Used lists

This commit is contained in:
Michael Käufl 2014-11-16 17:38:19 +01:00
parent 51dc9ddee0
commit b62e147e96
1 changed files with 16 additions and 12 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

28
notes.md
View File

@ -21,14 +21,14 @@ contain items already processed by the item handler.
The benefits of this include: The benefits of this include:
1) Batteries included: If the user is creating a program that 1. Batteries included: If the user is creating a program that
processes a set of feed urls that it repeatedly polls on an processes a set of feed urls that it repeatedly polls on an
existing feed then the built in database provides a "batteries existing feed then the built in database provides a "batteries
included" solution to prevent calling the users channel or item included" solution to prevent calling the users channel or item
handlers unnecessarily, greatly simplifying development of those handlers unnecessarily, greatly simplifying development of those
handlers. handlers.
2) Zero maintenance: The database is maintence free. There is nothing 2. Zero maintenance: The database is maintence free. There is nothing
the developer needs to do to take advantage of it. At the same time the developer needs to do to take advantage of it. At the same time
the developer doesn't need to query it at any time nor do they need the developer doesn't need to query it at any time nor do they need
to purge items from it. They need not even know it exists. to purge items from it. They need not even know it exists.
@ -70,23 +70,23 @@ from constantly refetching a bad feed url.
The current database solution contributes the following issues: The current database solution contributes the following issues:
1) The database is too simple: In the above example I need to track 1. The database is too simple: In the above example I need to track
(and prevent) the number of fetches I make for article items. I (and prevent) the number of fetches I make for article items. I
also need to allow a number of retry attempts before preventing also need to allow a number of retry attempts before preventing
refetches.The current database is not sophisticated enough to refetches.The current database is not sophisticated enough to
handle these two different cases at once. handle these two different cases at once.
2) The database does expire or clean up entries: I expect my program 2. The database does expire or clean up entries: I expect my program
to run for a very long time, processing many thousand feeds and to run for a very long time, processing many thousand feeds and
even more article urls. The current implementation of the database even more article urls. The current implementation of the database
is simple in that it continues to grow indefinately, consuming lots is simple in that it continues to grow indefinately, consuming lots
of memory. of memory.
3) The database replaces a job that is trivial to implement in a 3. The database replaces a job that is trivial to implement in a
handler: The current database doesn't provide anything that handler: The current database doesn't provide anything that
couldn't be developed by a user of the package with ease. couldn't be developed by a user of the package with ease.
4) The database doesn't provide a fine-grained enough key for 4. The database doesn't provide a fine-grained enough key for
duplicates: The current version uses a multitude of options for duplicates: The current version uses a multitude of options for
item keys and channels, all of which could very easily be falsly item keys and channels, all of which could very easily be falsly
marked as duplicates. For example, the item title is not a very marked as duplicates. For example, the item title is not a very
@ -107,19 +107,23 @@ package and moved to a set of built in handlers.
The developer will then be provided with two powerful options: The developer will then be provided with two powerful options:
1) Use the built in database handlers as part of a chain along with 1. Use the built in database handlers as part of a chain along with
their own existing handlers to get the same functionality as their own existing handlers to get the same functionality as
currently provided by the existing database implementation. currently provided by the existing database implementation.
2) Roll their own custom handlers which may be inserted into the 2. Roll their own custom handlers which may be inserted into the
handler chain. They even have the option of using the provided handler chain. They even have the option of using the provided
database handlers if they want. database handlers if they want.
This opens up exciting possibilities for developers and the future of This opens up exciting possibilities for developers and the future of
the go-pkg-rss package. We could add handlers for caching via the go-pkg-rss package. We could add handlers for
memcache, a retry count handler for channels, a whitelist/blacklist
handler based on item title, a filter handler that strips out items - caching via memcache,
that have a date older than 3 hours, etc. - a retry count handler for channels,
- a whitelist/blacklist handler based on item title,
- a filter handler that strips out items that have a date older than 3
hours,
- etc.
What's nice about this approach is that we can recreate a functionally What's nice about this approach is that we can recreate a functionally
identical approach that means this is backwards compatible with older identical approach that means this is backwards compatible with older