Revision history for StreamFinder
1.00 Date/time
First version, released on an unsuspecting world.
1.01 2018/08/10
- Fixed issue with some Facebook videos on Youtube.
- Removed Tunein as a valid site (can no longer extract streams).
1.02 2019/01/31
- Make StreamFinder::Youtube the default and handle pretty much
anything youtube-dl handles, ie. brighteon, vimeo, etc.
1.03 2019/08/15
- Fix StreamFinder::Reciva now failing to connect to server
(added wget as fallthrough as LWP::UserAgent and curl fail with
unsupported SSL protocol errors).
- Fix StreamFinder.IHeartRadio now failing to grab images / icons.
- Strip off "?autoplay=true" from StreamFinder::Youtube urls.
- Minor doc fixes.
1.04 NOT RELEASED
- Make debug option work for all stream types, and allow specifying
as either "debug", "-debug", "debug => 1", or "-debug => 1" in
parameter list for new()
(ie. my $station = new StreamFinder($url, -debug => 1);
- Better handle ".m3u8" streams returned by brighteon.com (Youtube).
1.05 2019/09/03
- Add StreamFinder::BannedVideo (https://banned.video/) to handle
their video streams (which youtube-dl doesn't currently handle).
1.10 2019/09/07
- Add user-configuration files (~/.config/StreamFinder/config,
~/.config/StreamFinder/<submodule>/config (see docs).
User can now specify / override most LWP::UserAgent options, ie.
"agent", "timeout", etc. in the new configuration files.
User must create these files, if needed with a text editor.
- Greatly reduce falling back to wget by adding default user-agent
and checking for wget availability. Specify an "agent" => "your agent"
line in the user-configuration file to change.
- Add "-random" and "-noplaylists" options to $station->getURL()
(see docs)
- StreamFinder::BannedVideo: Add "-keep" option to better control
type and order of streams returned (see docs).
- StreamFinder::Facebook: Depreciated ~/.config/.fbdata for new
user-configuration files.
- StreamFinder::IHeartRadio: Add podcast-playing capability.
- StreamFinder::IHeartRadio: Depreciated old stream order options,
replacing them with new -keep and -skip arguments and configuration
file entries.
- StreamFinder::RadioNet: (https://www.radio.net/) ADDED
- StreamFinder::Tunein: RESTORED (now uses StreamFinder::Youtube for
stream fetching), but adds metadata loading (which youtube-dl does
not currently fetch for these streams.
- StreamFinder::Youtube: Better handle "chunky" brighteon.com
.m3u8 video streams. Also add "-notitle" option, streamlining
use by StreamFinder::Tunein (see docs).
1.11 2019/09/07
- StreamFinder::Youtube: bugfix for brighteon.com m3u8 video URLs.
- Docs: fix a cpl. typos, actually include the updated README file.
1.12 2019/09/10
- StreamFinder::Tunein: Add podcast-playing capability.
(Will look for "playUrl" before falling back to youtube-dl for stream).
- StreamFinder::Reciva: Fix to make LWP::Useragent fetch reciva URLs
using the ANCIENT TLSv1 to allow to work without falling back to wget.
See the StreamFinder::Reciva docs for creating a one-line custom
config file to do this - I didn't make it the default since they could
FIX this at ANY time, which would break things!
- Add URI::Escape and HTML::Entities as DEPENDENCIES - in order to
ensure that the returned stream titles are un-escaped.
- Fix $station->getID('fcc') to work properly (return the station's
FCC call-letters for sights and stations that support it.
- Set $station->{artist} and $station->{year} where applicable.
- Minor doc updates.
- Fix some sight-specific submodules to properly accept just a station
/ video ID rather than a full URL (see docs).
- Add "desc" option go getTitle() to fetch station's (long) description
(for sites that provide that field), otherwise, the title is returned.
1.13 2019/09/11
- Fix missing (recently-added) file (RadioNet.pm) in MANIFEST file.
- More doc. fixes, mostly to the individual submodules.
1.20 2019/09/16
- Removed usage and all references to WWW::Youtube::Download -
This module no longer fetches the "author" (Youtube user) nor does
anything else we can't more easily do manually.
- StreamFinder::Apple: (podcasts.apple.com) ADDED!
- StreamFinder::Brighteon: (brighteon.com) ADDED (split off from
StreamFinder::Youtube).
- StreamFinder::Vimeo: (vimeo.com) ADDED (split off from
StreamFinder::Youtube.
We separated these for better support - ie. ability to get the "author"
fields and access some additional streams. We also can often avoid
having to call youtube-dl.
- StreamFinder::Youtube: Removed "notitle" option, added "fast" option,
and add "description" field fetch to youtube-dl. Fixed url-fetching
order to call youtube-dl FIRST to prevent infinite stall if called
with an already-valid stream-url (Fauxdacious mediaplayer).
StreamFinder::IHeartRadio - Fix sometimes-corrupted ID field.
StreamFinder::Reciva - Fix failure to always capture ID field if
given wrongly-formatted full url.
- Add $station->{'genre'} field to IHeartRadio, RadioNet, Reciva, and
Tunein (the sites that support it).
- Various small code and doc. tweaks / cleanups.
- DEBUG was accidently left set to ON, now turned OFF by default!
1.21 2019/09/18
- Significant documentation improvements & polishing.
- StreamFinder::BannedVideo: Fix incorrect IDs returned.
- StreamFinder::Tunein: (new()) - Fix mishandled URL if only
station-ID specified (non-full URL).
- StreamFinder::Apple: (new()) - Handle if only podcast-ID specified.
1.22 2019/10/07
- Minor bugfixes, namely handling double-quotes in titles in
some submodules.
- Minor doc. improvements.
1.23 2019/11/02
- StreamFinder::BannedVideo: Fix to handle change in their site -
to execute extra step now needed for infowarsmedia.com videos and make
the retrieved mp4 stream the "best" stream.
1.24 2019/11/06
- StreamFinder::Youtube: Eliminate express page scan for metadata for
non-Youtube sites processed to prevent ocassional infinite loop caused
by attempting to scan a media stream instead of a website.
- Unescape HTML entities in the Genre string in Reciva & Tunein.
- Minor doc. improvements.
1.25 2019/11/08
- StreamFinder::Brighteon: Fix another site-change that caused blank
titles to be returned, and broaden title and description searching
ability.
1.26 2019/11/19
- Correctly handle "HOME" directory in M#-Windows.
- StreamFinder::IHeartRadio - Handle non-US site URLs. Change default
"keep" priority to "secure_shoutcast, shoutcast, secure, any" for
easier use by Fauxdacious.
1.27 2020/04/23
- Added StreamFinder::Spreaker for https://widget.spreaker.com
podcasts.
- Minor doc. improvements.
1.28 2020/04/25
- Fix failure to include StreamFinder::Spreaker in MANIFEST and
to update change log.
1.29 2020/07/02
- Split out video's channel page (that was previously appended onto
the "artist" field) into a new, separate "albumartist" field. This
affects the video sites: Brighteon, Vimeo, and Youtube.
(Old way was: artist:="Artist - Artist's video channel URL"). These
are now put into separate fields.
- Fix .pls playlists returned by some Tunein stations by stripping off
some garbage Tunein appends in the format:
"stream.pls?DIST=TuneIn&TGT=TuneIn&maxServers=..." and more accurately
obtain title, artist, and album fields for stations vs podcasts.
- Fix issue causing some vimeo videos to fail to fetch stream.
- Clean up some "dirty" Spreaker titles (convert invalid characters).
- Marked StreamFinder::Radionomy depreciated - site appears defunct.
1.30 2020/07/05
- Add "-nopls" option to getURL() to parse ".pls" playlists and
return only a single entry, like "-noplaylists", but pass others
(".m3u" and ".m3u8") through unchanged. This needed because most
".m3u*" are playable "HLS" streams, whereas ".pls" playlists are
actually playlists and many players will not play them as-is.
- StreamFinder::RadioNet: Fix issue with no streams being fetched for
some stations & clean up titles a bit.
- StreamFinder::IHeartRadio: Add year to podcasts, fix some title
and description issues.
- StreamFinder::Spreaker: Fix some streams not working, fix "-debug"
option to work.
1.31 2020/08/10
- Add StreamFinder::Blogger (www.blogger.com) for Blogger videos.
- Add StreamFinder::Bitchute (www.bitchute.com) for Bitchute videos.
1.32 2020/10/16
- Remove StreamFinder::BannedVideo as banned.video has become so
paranoid as to add sophisticated anti-bot/scraping detection that I
have been unable to figure out how to circumvent.
NOTE/RANT: They claim to be pro freedom / anti-big tech spying, etc.
but their site has become unusable without third-party / Google
javascripts, and other malware nasties we and our users are trying to
avoid for our own safety! Most of their videos can now also be
found on Bitchute anyway!
- StreamFinder::Youtube: Fix minor bug that sometimes returned no
streams.
1.33 2020/11/01
- StreamFinder::Apple: Fix some icon images not being downloaded.
- Add StreamFinder::SermonAudio.
- Remove depreciated StreamFinder::Radionomy (site now defunct).
1.34 2020/11/02
- Add StreamFinder::Castbox (https://castbox.fm podcasts)
- StreamFinder::Spreaker: Enhance to accept podcast page URLs (like
other podcast-ish modules rather then just the bare widget.spreaker...
URLs. These pages also have artist and albumartist fields! Also,
properly return the total no. of streams found ($podcast->count()).
- "internationalize" several sites (eliminate the ".com" from the
required matching regices (allow for ".eu", ".net", etc. to match").
1.35 2020/11/04
- Fix regex glitch breaking the sites we "internationalized" in v1.34.
1.36 2020/11/16
- StreamFinder::Apple: Return correct description for podcast episodes.
- StreamFinder::Spreaker: Fix minor title and image issues.
- StreamFinder::Youtube: (For now): convert www.youtube.com to
youtube.be to get around youtube.com seeming to block youtube-dl.
1.37 2020/11/25
- Add StreamFinder::Rumble for videos on rumble.com
- StreamFinder::Castbox: Return full description for podcast episodes.
1.38 2021/01/01
- StreamFinder::IHeartRadio: Handle streamless custom radio-station
urls, ie. "https://station.iheart.com" (revealed by CPAN bug# 133982).
These reference the correct "live" url that contains the steams!
Also fix description, artist and albumartist fields failing to return
values for (some?) podcasts.
- StreamFinder::Rumble: Strip the podcast ID field to just it's
unique ID.
- StreamFinder::Reciva will likely be removed next release after
01/30/2021 due to that site's announcement to go offline after
that date!
1.40 2021/01/04
- Add StreamFinder::Google for Google (podcasts.google.com) podcasts.
(closes CPAN feature# 133993).
1.41 2021/01/06
- StreamFinder::Google, StreamFinder::Castbox, StreamFinder::Spreaker,
and StreamFinder::IHeartRadio: Add ability to accept a podcast /
channel page without a specific episode and return the first (latest)
episode for that page. NOTE: StreamFinder::Apple already does this
but returns all episode streams in a list with the first episode the
first stream, but the metadata returned is for the Apple podcast page,
and left this way for backward compatability.
1.42 2021/04/11
- StreamFinder::Rumble: Fix failure to find any streams (Rumble
chgd. their html slightly), and Add "-keep" option to limit / order
the streams fetched (default: mp4,webm,any) - see updated docs.
- StreamFinder::Apple: Remove music.apple.com (music samples no
longer scrapable? (Much more complicated and I never use it - you
only ever could get short sample clips - not full songs anyway!
Also, misc. fixes/tweaks, also see last item below:
- StreamFinder::Castbox: Added Album field.
- StreamFinder::Spreaker: Fix failure to fetch anything (Spreaker
chgd. their html a bit!). Also, added Album field, and additional
check for possible AlbumArtist url.
- StreamFinder::Google: Misc. fixes/tweaks, also see below:
- Apple, Castbox, and Google: Add ability to fetch the first (latest?)
podcast, and return all podcasts as an extended MP3 playlist
(new "-playlist" option to the $station->get() function) - for podcast
author's main page urls.
1.43 2021/04/11
- StreamFinder::Rumble: Fix doc glitch in SYNOPSIS section specifying
the "-keep" option as a hash reference (must actually be an array
reference, causing user to not get any streams if using the SYNOPSIS
example as a program. Also fixed this in StreamFinder::IHeartRadio!
- StreamFinder::Spreaker: Fix failure to fetch icon image for some
podcasts.
1.44 2021/04/19
- StreamFinder::Rumble: If only a podcast "ID" is given, check to see
if it's either the long ID for the html page or the short one used for
the embed page and append the appropriate url (prev. only the long
html page ID string was supported. Either way, the correct Rumble ID
will be returned.
- Add some default values for "genre", such as "Podcast" for Spreaker,
Castbox, Apple, and "Video" for Youtube, Vimeo, Brighteon and Bitchute.
- Initialize some variables that should be.
- StreamFinder::Apple: Better checks for album, genre and year.
- Misc. doc-fixes all over.
1.45 2021/05/04
- Add StreamFinder::Anystream - as a "fall-back" module to search any
specified webpage URL for streams (streams are matched by searching
for URLs matching specific known stream types, ie. ".mp3", ".ogg", etc.
(The default "-keep" list is: "mp3,ogg,flac,mp4,m4a,mpd,m3u8,m3u,pls").
- Added StreamFinder option "-omit" to specify submodules that you
have installed to NOT be used, ie. -omit => "Anystream,Youtube".
1.46 2021/05/05
- Oops - add StreamFinder::Anystream to MANIFEST.
- Add & document new -secure option (all) to allow user to limit
streams returned to (secure) https streems.
- Minor doc. cleanups.
1.47 2021/05/10
- Fix potential infinite loop if unrecognized option arg. passed in.
- StreamFinder::Youtube - Add fallback ability to check html page for
embedded iframe containing a valid StreamFinder page, and new option:
"-noiframes" to not do this (if set to 1/true).
1.48 2021/05/19
- StreamFinder::Youtube, StreamFinder::Anystream - Only look for
streams in actual HTML pages, NOT media URLs - to prevent potentially
very long or even infinite delay trying to fetch the URL.
- StreamFinder::Spreaker - Unescape artist and album fields, remove
some extraineous debug output.
1.50 2021/05/22
- Refactor all modules to combine all common methods into a separate
internal module: StreamFinder::_Class (new) to greatly simplify and
eliminate redundant code. This should make creating new site modules
much simpler (now most if not all functions other than "new()" should
not be required in site modules unless it's necessary to override)!
This better follows proper object-oriented programming standards.
- StreamFinder::Tunein - Add "-notrim" option to NOT trim extra "ad",
misc. stuff from stream URLs (previously, and by default URLs are
"trimmed" removing this stuff (after and including the first "?").
- StreamFinder::Tunein, StreamFinder::IHeartRadio - Attempt to grab
full description of podcasts (may include HTML, ie. links).
- Misc. minor code cleanups found while converting.
- StreamFinder::Anystream - Fix bug that mangled some streams returned.
Also make sure URLs returned containing "\/"s are unescaped to "/"s.
1.51 2021/07/01
- Add "-log" and "-logfmt" options to append a line to a specified
log file for each successful stream-url fetch (see docs).
- Fix bug that caused config file arguments with unescaped Perl regex
characters in their values to be ignored.
- StreamFinder::Rumble - Fix bug that caused description to be cut
short.
- StreamFinder::Spreaker - Handle their multiple URLs, and also fetch
video streams.
- StreamFinder::SermonAudio - Handle additional "alternate" URLs as
well as video streams they can contain. Also try harder to find icon.
1.60 2021/10/02
- Add 'artist' option to getIconURL() and getIconData() functions,
(Currently only Apple, Brighteon, and Youtube) to return the "channel"
(AlbumArtist) icon instead of the individual podcast icon. We may add
this to more of the other sites later, but this will likely require an
additional, separate fetch of the channel page.
- Small doc. touchups.
1.61 2021/10/03
- Add 'artist' icon option (see v1.60 changes) to StreamFinder::Rumble,
StreamFinder::Castbox, StreamFinder::Spreaker and StreamFinder::Vimeo.
1.62 2021/10/08
- Fix getImageData() returning icon instead of image if wget used to
fetch.
- getImageUrl() and getImageData() can now also accept the 'artist'
argument and return a separate artist / channel image, if one larger /
different from the corresponding icon exists. Currenty, only
StreamFinder::Odysee provides these.
- Add StreamFinder::Odysee to handle odysee.com videos
(requires youtube-dl).