-- Nov 18 In-Class Exercise Thread
Keyword Advertising:
Data consolidation - We want to get information from external source systems and integrate them into the currentdata store. This a useful method of allowing multiple advertisers on to the result page. We can use data consolidation to keep track of data such as which keywords on which they are advertising, how much they are paying for our service, an the price at which they sell their products. There is a high latency to this pattern, so it should be used sparingly (preferably once); thus, a kind of data pipeline should be established so changes are quickly reflected, which leads us to...
Changed Data Capture - If there are updates to the external source's tables (say, a change in the website's description or a list of products that are seen on the advertisement result page), we want them to be reflected in our consolidated data store. Say a certain product suddenly becomes very frequently searched; it would be good to be able to implement something like surge pricing i.e. increasing the price in accordance with greater popular interest. This allows for more timely advertisements.
Product Suggestion:
Data federation - While the pull approach here is rather slow, product suggestion queries tend to be simple enough that they are likely to perform better than others. The reason we want to use Data federation is because it allows for products from different data warehouses to be displayed, and we will want to be querying on certain datapoints such as genres, tags, price, item type, etc. Seeing as we want these queries to be even faster, it would be helpful
to add a cache of some kind, so...
Data virtualization - If a user is looking at products of specific types, maintaining a cache which stores products keyed to their interests (datapoints like genre, price, item type, etc. that can filter out uninteresting items) would be an extremely useful tool to retaining customers.
(
Edited: 2020-11-18)
Keyword Advertising:
Data consolidation - We want to get information from external source systems and integrate them into the currentdata store. This a useful method of allowing multiple advertisers on to the result page. We can use data consolidation to keep track of data such as which keywords on which they are advertising, how much they are paying for our service, an the price at which they sell their products. There is a high latency to this pattern, so it should be used sparingly (preferably once); thus, a kind of data pipeline should be established so changes are quickly reflected, which leads us to...
Changed Data Capture - If there are updates to the external source's tables (say, a change in the website's description or a list of products that are seen on the advertisement result page), we want them to be reflected in our consolidated data store. Say a certain product suddenly becomes very frequently searched; it would be good to be able to implement something like surge pricing i.e. increasing the price in accordance with greater popular interest. This allows for more timely advertisements.
Product Suggestion:
Data federation - While the pull approach here is rather slow, product suggestion queries tend to be simple enough that they are likely to perform better than others. The reason we want to use Data federation is because it allows for products from different data warehouses to be displayed, and we will want to be querying on certain datapoints such as genres, tags, price, item type, etc. Seeing as we want these queries to be even faster, it would be helpful
to add a cache of some kind, so...
Data virtualization - If a user is looking at products of specific types, maintaining a cache which stores products keyed to their interests (datapoints like genre, price, item type, etc. that can filter out uninteresting items) would be an extremely useful tool to retaining customers.