|
|
i---------------------------------------- |
|
|
|
iIdeas for an aggregator |
|
|
|
iFebruary 06th, 2018 |
|
|
|
i---------------------------------------- |
|
|
|
i |
|
|
|
iI want to build a new gopher aggregator, specifically for phlogs, |
|
|
|
iand I want it to work in a very particular way. I'm not sure this |
|
|
|
iis going to be a general solve for everyone, but if I can do it |
|
|
|
iright it will work for my personal use. |
|
|
|
i |
|
|
|
iVersion 1 Plan: |
|
|
|
i |
|
|
|
i - It should know that a phlog has been updated, not a whole |
|
|
|
i gopherhole. |
|
|
|
i |
|
|
|
i - I should provide it an array of gophermaps to keep track of. |
|
|
|
i |
|
|
|
i - The gophermap will be hashed each day--store the hashes in |
|
|
|
i a file for simplicity, or a sqlite db-- and if the hash |
|
|
|
i changed from the previous day, that link is added to the |
|
|
|
i daily list. |
|
|
|
i |
|
|
|
i - If the link doesn't resolve, ignore. It might be temporary. |
|
|
|
i |
|
|
|
i - It should organize phlog updates by date so I can go back if |
|
|
|
i I missed a couple entries (like Bongusta) |
|
|
|
i |
|
|
|
i - It should auto-update daily |
|
|
|
i |
|
|
|
i - Output should be simple: |
|
|
|
i |
|
|
|
i - Master gophermap is a symlink to the current month. |
|
|
|
i |
|
|
|
i - Each gophermap is a year-month for easy archival |
|
|
|
i |
|
|
|
i - Viewing the directory will list the archive and current |
|
|
|
i |
|
|
|
i - Linking to current will include a link to the archive |
|
|
|
i (directory view) |
|
|
|
i |
|
|
|
iVersion 2 Plan: |
|
|
|
i |
|
|
|
i - The link added to the aggregated list should point to the new |
|
|
|
i entry, not the main gophermap. |
|
|
|
i |
|
|
|
i - Potentially do a diff on the gophermap if the hash has |
|
|
|
i changed and try to identify new links? |
|
|
|
i |
|
|
|
i - Grab any new links and drop them in verbatim, potentially |
|
|
|
i |
|
|
|
i - Relative links might need to be rewritten |
|
|
|
i |
|
|
|
i - Problems with changed links? |
|
|
|
i |
|
|
|
i - Would this be instead of linking to the map, or in addition |
|
|
|
i (sub links under a heading?) |
|
|
|
i |
|
|
|
i - Allow a way for people to submit their phlog to the list and |
|
|
|
i make it a public system instead of a private aggregator |
|
|
|
i |
|
|
|
iMy plan was to implement this in Perl as a way to learn Perl. |
|
|
|
iI may still go ahead with that, but it might be easier for me to |
|
|
|
ibuild with my current knowledge in python. Regardless, if anyone |
|
|
|
ireading this wants to take a stab at it, go right ahead. I'd |
|
|
|
irather have the tool than be the one to write it. |
|
|
|
i |
|