c# - Possible Ways to Merge/Join Similar Collections from Different Systems -


i keep running same programming task without satisfying solution. have similar collections of objects different systems need combine or merge one, , possibly report on intersections between two.

a example of might collection of users active directory, , same collection of users sap (with richer attributes not exist in ad). want 1 collection of users containing properties both collections.

or maybe have collection of users in sharepoint, , collection of newsletter subscribers in constant contact, , want collection of active users newsletter subscribers in constant contact.

given there common identifier in both collections (email address, id of sort) join them, find have few options efficiently merged data:

  1. get objects system a. objects system b. in double-loop, find matches , add them new collection.
  2. get objects system a. each object in system a, query system b find match , add new collection.

option 1 stinks because have fetch data system b, though may throw of away if there no matches. option 2 stinks because i'll have many individual queries system b matches.

i know can setup kind of cube processes stuff regularly, seems should able take 2 collections, denote common piece of data between them, , ask framework join me intelligently. there other method may missing here?

thanks, adam

mathematically either have or b in fashion- there no getting around it.

the typical optimization close either or b possible, e.g. copy data b , ask b's database non-matching elements, or effect. choice of system copy can based on technical considerations (e.g. closed or inaccessible systems mainframes copied from), performance considerations (b may faster or more scalabale system a), or data size considerations (if a's data size order of magnitude smaller b's, makes more sense copy b rather vice versa).

if data sources can produce ordered streams of data, can perform comparison in streaming fashion, rather needing of data both systems. instance:

a's data     b's data            b            c c            d d            f e f

if know data sorted, can iterate both lists looking matches, rather doing look-ups against 1 data source.


Comments

Post a Comment

Popular posts from this blog

apache - Add omitted ? to URLs -

i working on website cms reads page name of query string, e.g. http://exmaple.com/?pagename . trying create rewrite rule rewrite ? . so, user types in /pagename , taken /?pagename try this: rewriteengine on rewritecond %{request_filename} !-d rewritecond %{request_filename} !-f rewriterule ^/?(.*) index.php?$1 [l]
Read more

redirect - bbPress Forum - rewrite to wwww.mysite prohibits login -

i using bbpress installation without wordpress integration. the redirect of rewriteengine @ forum site works: rewriteengine on rewritecond %{http_host} ^forum.mysite.com$ rewriterule (.*)$ http://www.forum.mysite.com/$1 [r=301,l] the problem login pppress. bbpress login not recognize: http://www.forum.mysite/bb-login.php . it redirects to: http://forum.mysite.com/bb-login.php . i tried permalink settings none , name based, same issue. does uses redirect in bbpress or how can eliminate "double" content , redirect permanently www.? i redirected www.forum.mysite.com forum.mysite.com , achieved eliminate "double" content.
Read more

php - How can I stop spam on my custom forum/blog? -

so have custom built forum & blog system of late has been dealing lot of spam. if wordpress use akismet, if different common platform i'm sure i'd find plugin. there kind of static class can download this? using php. i'd still go akismet, if it. uses outside of wordpress, may have pay fee it, depending on use -- check terms , conditions -- it's option , easy implement in php using api. use api key wordpress. com account access. basically, grab whichever php client library takes fancy -- use alex potsides' php5 library -- plug in key, , it's handful of lines of code. here's bare bones of validation straight 1 of live sites: ... if ($akismet) { $akismet->setcommentauthor($name); $akismet->setcommentauthoremail($session->userinfo["email"]); $akismet->setcommentauthorurl(""); $akismet->setcommentcontent($sentence);...
Read more