Tech Forum Archive

Membership vCards with Photos

28 postsstarted by mkmurray 19,093 viewsoriginal topic

mkmurray
Senior Member
#1
My plea still stands for the developers to include a checkbox or something for adding photos into the vCard export that is now available:

http://tech.lds.org/forum/showthread.ph ... #post17698

In the meantime...

I've started writing a little .NET application that will be able to authenticate with LUWS using user-entered credentials and scrape the photos off of the user's local unit Membership Directory. Then the application will put the binary photo data in the appropriate vCards.

The next step (for Outlook users) is to split the one vCard file into separate files. I wrote a little piece of code in about 10 minutes that does this, which I may or may not include the above application's functionality. However, I found a blog post where a user wrote some very helpful VBScripts, one of which does this. I will discuss point you to this in the next paragraph.

So there is one problem with importing these vCards with embedded photos into Outlook (or Windows Contacts, as found in Vista if you don't have Outlook): this problem is that every test I have run as failed to import the picture in the vCard into Outlook. Apparently Outlook doesn't look for the "PHOTO" property or something. So I found the following blog post where somebody created a fix that I am going to try out:

*EDIT: I found out this only works for getting vCards with Photos into Exchange Server (more of an enterprise use than a personal, home use). Read the next post for more info.*

http://gsexdev.blogspot.com/2005/09/imp ... es-to.html

He gives a zip file to download with VBScripts in them which I have attached to this post (one of these scripts can do the vCard splitting from one .vcf file to separate .vcf files). In his blog post, he explains how to configure and use the scripts on your computer.

I will keep this thread posted on how my application is going and my quest to get the Ward Photos on my phone (and hopefully your phone/PDA).

[Attachment Removed; read next post to find out why]
mkmurray
Senior Member
#2
mkmurray wrote:...

So there is one problem with importing these vCards with embedded photos into Outlook (or Windows Contacts, as found in Vista if you don't have Outlook): this problem is that every test I have run as failed to import the picture in the vCard into Outlook. Apparently Outlook doesn't look for the "PHOTO" property or something. So I found the following blog post where somebody created a fix that I am going to try out:

http://gsexdev.blogspot.com/2005/09/imp ... es-to.html

He gives a zip file to download with VBScripts in them which I have attached to this post (one of these scripts can do the vCard splitting from one .vcf file to separate .vcf files). In his blog post, he explains how to configure and use the scripts on your computer.

...

Unfortunately, this VBScript to import vCards with embedded photos only works for Exchange Server, not the local desktop Outlook client. It puts the contacts on the global Exchange Server contacts.

I am forced to find another way to get these into Outlook. I suppose I could learn the .contact file format, as I have looked at it when it has an image embedded in it. Perhaps I could write something to auto-import all vCards into Outlook and then go back and attach photos after they are in. This will be tough to do in code, but I think I'm gonna give it a shot.

Again, I'll keep you posted.
johnshaw
Senior Member
#3
As I investigated this for my own purposes I found that the windows mail client (used to be called outlook express) did support the vcard format fully and the contacts imported just fine with the one file. you could then use Outlook to pull in the contacts from the mail client. I'm not a big fan of having to do this multiple times, and I've made my bread and butter on Microsoft, but the Outlook vcard issue is ridiculous.
“A long habit of not thinking a thing wrong, gives it a superficial appearance of being right, and raises at first a formidable outcry in defense of custom.”
― Thomas Paine, Common Sense
mkmurray
Senior Member
#4
jshawut wrote:As I investigated this for my own purposes I found that the windows mail client (used to be called outlook express) did support the vcard format fully and the contacts imported just fine with the one file. you could then use Outlook to pull in the contacts from the mail client. I'm not a big fan of having to do this multiple times, and I've made my bread and butter on Microsoft, but the Outlook vcard issue is ridiculous.
Thank you, I didn't try the multiple vCards in one file thing with the Windows Mail client, I only tried it with the latest Outlook and figured they would use the same algorithm. However, I did import one vCard into Windows Mail (just to get it into Windows Contacts) and the embedded image did not come through with it.
mkmurray
Senior Member
#5
mkmurray wrote:I will keep this thread posted on how my application is going and my quest to get the Ward Photos on my phone (and hopefully your phone/PDA).
My efforts are going up in flames quickly. This is not an easy task. The more I tread into pretty sloppy parsing based on the hope that the Church won't change the HTML structure, the more uncomfortable this is becoming.

Let me explain what I have found, as this may be interesting to the Church developers over LUWS...

With my application, I have written a series of code that logs into your Ward Website and begins scraping off the info of each member in the ward with their accompanying family photo. My application also downloads your ward's vCard export from the LUWS site.

Now in order to match up photos with the family vCard, I have to find a piece or set of data to join on. Unfortunately, the vCard entries all have UID properties that seem to come out of thin air. That would be the best piece of data to join on, whatever it is. However, it is not found anywhere in the HTML source of your ward's Membership Directory in LUWS.

So the next best thing to ensure you have the right vCard is to check every other piece of data (names, address, phone, email). This is definitely the hard way and pretty thorough, but it proved possible and so I went forward. However, the vCard export data and the data you can scrape out of the HTML on the LUWS Membership Directory often can differ. Let me enumerate some cases:
  1. The already available LUWS vCard export includes as many Given Names as it can for an individual; the LUWS Membership Directory HTML lists at most 2 Given Names.
  2. The LUWS Membership Directory HTML leaves off all suffixes (Jr., II, etc.); the LUWS vCard export does not leave them off.
  3. The LUWS vCard export enforces a "proper casing" (first letter capital, all other letters lowercase) on all words in a name; this makes names like "McKenzie" and "ten Bosch" appear as "Mckenzie" and "Ten Bosch", respectively. It even formatted the suffix "II" to be "Ii". The LUWS Membership Directory HTML leaves casing as is and applies no such formatting.
  4. The name displayed in both the vCard export and the Membership Directory HTML may possibly have an extra space before or after it; I believe the source of this is that some of the registered members on LUWS typed in their Preferred First Name and left a leading or trailing space in the text field. Unfortunately, the code that puts together the vCard export doesn't trim any leading or trailing spaces like this, but the code I wrote to scrape the Membership Directory HTML does. Thus, a mismatch is occuring. Plus, the vCard properties now have extra spaces in some of the properties, which is actually kind of annoying if your Contacts program or PDA/Cell Phone doesn't parse the extra space out.
I could try to work around these problems, but it just makes this whole thing feel even sloppier. The real solution would be to have the Ward Photos embedded or linked to somehow in the vCard export. I suppose I will list this on the LUWS Suggestion List on the LDS Tech Wiki.

The vCard standard supports either doing this with embedded Base64-encoding binary:

Code: 

PHOTO;ENCODING=BASE64;TYPE=JPEG:/9j/4AAQSkZJRgABAQEASABIAAD/2wBD...

...or as a URL:

Code: 

PHOTO;VALUE=URL;TYPE=JPEG:[URL]https://secure.lds.org/units...blahblahblah...QwNjA5,00.jpg[/URL]
Pretty please, Church developers? :D

If you can at least connect the photos to the vCards (or even just put the same UID property from the vCards into the Membership Directory HTML, maybe like the "id" of the table containing the family or something), then I could continue and build something to auto import these vCards into Outlook with their photos attached to them.
mkmurray
Senior Member
#6
mkmurray wrote:My efforts are going up in flames quickly. This is not an easy task. The more I tread into pretty sloppy parsing based on the hope that the Church won't change the HTML structure, the more uncomfortable this is becoming.
For now, I'm just using the MLS exports, as that gives me the most bang for my buck. I wish I could figure out how to get photos linked with the MLS exports...

Anyway, the method I'm using is Windows Contacts, as I have confirmed that Windows Mail (Outlook Express's successor within Vista) can import a single .vcf file containing multiple vCards. It's terrific the amount of info you get in those MLS exports (HT/VT, birthdays, callings, etc.). Really helpful to my calling of Membership Clerk.
kennethjorgensen
Community Moderators
#7
mkmurray wrote:My efforts are going up in flames quickly. This is not an easy task.......

If you can at least connect the photos to the vCards (or even just put the same UID property from the vCards into the Membership Directory HTML, maybe like the "id" of the table containing the family or something), then I could continue and build something to auto import these vCards into Outlook with their photos attached to them.
Can I emphasise and support this point too. Having ID's exposed for any records (families, individuals etc) would be a HUGE benefit for those of us who wish to write some added utilities to support the church's existing applications.
mkmurray
Senior Member
#8
mkmurray wrote:So there is one problem with importing these vCards with embedded photos into Outlook (or Windows Contacts, as found in Vista if you don't have Outlook): this problem is that every test I have run as failed to import the picture in the vCard into Outlook. Apparently Outlook doesn't look for the "PHOTO" property or something.
So I found out something intersting about Outlook 2007. If I add the embedded photo into the .vcf file and then also change the vCard version number to be 3.0 (instead of 2.1), then the photo will import correctly. However, this only works for Outlook 2007; with 2003, it appears the photo will not correctly import regardless of the vCard version number (I only tried 2.1 and 3.0).

I may pick up this project again and try to solve the problems I named before by doing a "best match" algorithm. It would try to verify all pieces of information about the household and rate the match by percentage or score. If it is 100%, then there is no reason to show this to the user. If there isn't a 100% match, the I would show the user all matches above a certain percentage (like 80% or 90%) and let them tell me which is the correct match.

I personally would like to be able to match up the LUWS photos with the MLS vCards as well (as there is so much more pertinent info for my calling in those vCards), but perhaps that will be a version 2.0 thing or something, as it would only benefit those that can reasonably acquire the MLS exports.

Again, I'll keep everyone posted...
mkmurray
Senior Member
#9
mkmurray wrote:I may pick up this project again and try to solve the problems I named before by doing a "best match" algorithm.
I did indeed pick up this project again. I was able to use the Levenshtein Distance algorithm to get a pretty good "best match" algorithm (see http://en.wikipedia.org/wiki/Levenshtein_distance). This algorithm will eventually help with matching MLS vCards to the photos, because there will probably be more discrepancies between MLS data and LUWS data (specifically preferred names and email addresses).

For now though and from what I have found from my preliminary test of a couple wards, the best match of each household between the LUWS html and the LUWS vCards is no less than about 93%. If the match is 100%, there could be a second best match as high as like 80%; but if the first match is 100%, there is no reason to consider a second match. If the match is less than 100% (like 93%), I've almost always seen the second best match be like 45%. In that case, it's pretty safe to assume the top match is it. If the first 2 or 3 matches are pretty high and the top one is not 100%, I have designed a UI that the user can visually see the choices and percentage match, and then make the correct match themself.

Anyway, I need to clean up some of the UI and do some better exception handling. Otherwise, the app can successfully match up the photos to the vCards from LUWS. My current vCard output works perfectly with Mac's AddressBook. I'm working on a way to get it into Outlook 2007, the problem being Outlook won't import more than one vCard at a time. Windows Mail (and it's use of Windows Contacts) can import more than one vCard at a time, but I can't get it to import the photos like Outlook will (if I change the vCard version number to 3.0). Outlook 2007 is then able to import contacts from Windows Contacts. So that is the solution if you want the vCards imported all at once without photos, as has been asked on a few other threads. I'll post that solution to the appropriate threads when I get a chance.

And as I said before, I would like to be able to match up the photos to the MLS vCards. So I've still got quite a bit of work ahead of me. I'll keep ya updated.
mkmurray
Senior Member
#10
I just wanted to let people know I made quite a bit of progress on this a while back, but have been so busy the last month or so with Tithing Settlement preparations and deadlines at work.

The project really is somewhat close to an initial release but I need to clean up the code some first. Also I'm trying to figure out how the best way to deploy Office plugins are (I have written a plugin that will import more than one vCard in a single .vcf file; as far as I can tell mine would be the only free version I can find of such a plugin).
nathan.ellsworth
New Member
#11

Any updates?

I cannot believe there isn't already a solution to this problem and you've written the first free multi vcard to outlook plug-in.

Pictures would be nice, but I'd be happy just with an easy way to go from LUWS vcard or csv download right into Outlook to then use ActiveSync to get the ward list into my Windows Mobile Treo 800w. Pictures also would just be icing. Is there some obvious easy method I've overlooked? It was a piece of cake to do with my old PalmOS device. Surely there are many members using WinMo who might have already figured this out?

Made any progress on any of this since the last post?

Thanks
mkmurray
Senior Member
#12
nellswor wrote:I cannot believe there isn't already a solution to this problem and you've written the first free multi vcard to outlook plug-in.

Pictures would be nice, but I'd be happy just with an easy way to go from LUWS vcard or csv download right into Outlook to then use ActiveSync to get the ward list into my Windows Mobile Treo 800w. Pictures also would just be icing. Is there some obvious easy method I've overlooked? It was a piece of cake to do with my old PalmOS device. Surely there are many members using WinMo who might have already figured this out?

Made any progress on any of this since the last post?

Thanks
Yeah, I'm sorry...I just haven't dedicated the time to finishing this up yet. I'll try to get back on the ball again and get this distributed to people, as I think it would be quite helpful.

I hope to make the program so that you can decide if you want the photos embedded in the vCards or not. One issue that put the project at a stand-still is that I've written a very simple Outlook plugin, but I'm not sure how to distribute it. You have to have signed key or something and I can't figure out if I can just create the key on my box with a .NET executable, or if I have to buy a legal one from Microsoft.

This last issue is the real reason that progress was nearly halted on this project. If someone has experience in creating Outlook plugins using Visual Studio (I think the framework is called VSTO, or Visual Studio Tools for Office), then please give me some guidance. If I solve my problem on my own, I will report back my findings.

The good news is that I found a site that does free Subversion hosting and project tracking. But the difference between this site and other sites you may have seen already is that the creator of this one insists that this free offering is a lifetime guarantee and you don't have to make your projects public either (they can remain private). You currently have a disk quota of around 1.5 gigabytes, which is not bad in my opinion. Here is the link to the site: http://xp-dev.com/
johnshaw
Senior Member
#13

Hoping for Good News!

Any success in distributing the code? With out latest Tithing Settlement we took family pictures, this would be a great addition to our resources as ward leaders!
“A long habit of not thinking a thing wrong, gives it a superficial appearance of being right, and raises at first a formidable outcry in defense of custom.”
― Thomas Paine, Common Sense
mkmurray
Senior Member
#14
jshawut wrote:Any success in distributing the code? With out latest Tithing Settlement we took family pictures, this would be a great addition to our resources as ward leaders!
I appreciate you guys keeping me honest! :) Sometimes I need to know people are depending on me to finish.

I think I have solved the issues with the Outlook add-in. I'm not willing to pay hundreds of dollars to get a digital identity verification certificate from a vendor like Verisign. So when you install the Outlook add-in, it will say it's from an Unverified Publisher. I just want to make sure it doesn't bring up that "do you trust this plugin" dialog every time you load Outlook though.

Also, I think I have decided to go with Microsoft's ClickOnce deployment, instead of using MSI installers. The reasoning is partly because I wanted to learn the deployment technology, and the other part is because ClickOnce has built into it the idea of automaticaly checking for updates and downloading the latest from the server.

Unfortunately, this kind of deployment can't really be done on any open source hosting websites from what I can tell. If anyone knows of a free hosting site that will support ClickOnce deployment, please let me know. I mean basically you just need a place to put files I think; however, if the hosting is being done with IIS, you have to add a new MIME type in the IIS settings. I have a contact that will let me host it for like $5 a month (and also give me the ability to configure IIS) or I have a server here at home that I could host it on. But honestly, either option I choose, will cost me money and possibly require me to babysit the server to ensure uptime. I may have to put up a donate link on the site I'm hosting the utilities on.

As for timeline of when I can get this up and distributed, I will try and double my efforts. Perhaps I can get something up by the end of this week or by some time next week. Thanks for your patience.
seanmikkelsen19-p40
New Member
#15
That is great to hear mkmurray! I have been watching this thread for months after I stumbled upon it in search of a solution to the same problem.

I did develop a very simple application that will split the vcard that you download from the church website allowing you to then import them into outlook with a macro type script. An outlook plugin would be amazing to have.

I too could not find any logical way of connecting the image file names with their vcards in order to include images. It sounds like you have that all figured out though.

I look forward to seeing what you have come up with!

Thanks for your efforts, it should help a lot of people.


I meant to include a link to my little app if anyone wants to play with it. I just hosted it on an existing website of mine. You can download it here:
http://www.airsoftgunhub.com/page/274452203/vcard


Just a quick explanation of what this does; it is a simple gui that allows you to browse for a .vcf (Vcard) file (one that contains multiple records in one .vcf file). You can then choose a save location, and split the file. This basically takes the vcard that you downloaded from the church website (or any other vcard you may have this problem with) and separates them into individual vcards (by family). It then gives you the option to import them into outlook. This is not however integrated into Outlook by any means as it is a nightmare trying to even begin figuring that out (kudos to you mkmurray!). So by import, what it does is simulates you opening the file, and saving it. This is a lot more sloppy than a plugin, but I got desperate for a solution to this vcard/outlook problem. I wrote this on vista running outlook 2007 with no problems. I did however notice that when trying to import the vcards (using my app) it usually ran much smoother to actually have Outlook open and running before doing so.

I would be interested to see if this is of any use to anyone else. In the meantime, I'm really looking forward to mkmurrays release, it should be great.
mkmurray
Senior Member
#16
Thanks a lot for your vote of confidence. I think I'm getting close to releasing the Outlook plugin. But as for the utility that links photos to vCards on the Ward Website, the code needs to be cleaned up some and verified that it works with the new LDS Account authentication system.

Unfortuantely, I've been home-ridden and sick nearly this whole week as I developed strep throat. I'll have to see if I can get back on it early next week.
seanmikkelsen19-p40
New Member
#17
I hate to be a pest, I know you have been sick, but any update?

Thanks,
Sean
mkmurray
Senior Member
#18
seanmikkelsen19 wrote:I hate to be a pest, I know you have been sick, but any update?

Thanks,
Sean
*sigh* Seriously...no offense taken at all...I need to be held accountable to this project! :) It's so close to being done.

I'm actually on vacation right now for the week. And work was crazy the last week in preparing to take this vacation and get the time off. I think I will be able to work on it again this next week. My new goal will be to have something posted during the next week (perhaps the Outlook plugin first).

Thanks again for the bump and reminder. It is sincerely appreciated.
mkmurray
Senior Member
#19
mkmurray wrote:My new goal will be to have something posted during the next week (perhaps the Outlook plugin first).
Ok, time for some updates because I need to make excuses for why this isn't released yet...

The Outlook plugin is functional and bug-free as far as I can tell. However, distributing the add-in is proving difficult. I decided I would learn the ClickOnce technology that Microsoft offers for distributing the add-in, especially for it's benefit to be able to check a URL on a regular schedule for the latest updates without me writing a lick of code. This install technology I have chosen as opposed to writing a regular MSI installer.

Things are going great except for security and trust in the ClickOnce world. The right way to do this is to shell out hundreds of dollars for a secure certificate from a reputable and trusted certificate authority (like Verisign, etc.) to sign my add-in with. So now I got to figure out a way for my installer to add my URL as a trusted site in the Internet Zone trusted lists stuff. I will keep you posted but this will set me back a few days (it already has taken me a few to figure out the cause and possible solutions to the install error I am getting).

A few weeks ago, I rented some hosting space from one of our fellow community members (jcircus) for a decent monthly rate. It's also one of the few hosting services that allowed me to add custom MIME types to IIS for enabling this ClickOnce type of installing. So the good news there is that I already have a place to host this once I get the install bugs worked out.

Thanks for your patience, everyone.
RossEvans
Senior Member
#20
When this application is released, will users have access to your final vCard file (including embedded photos), or will they have to use Outlook? I might want to load the portable .vcf file into some other platform, but I have no plans to upgrade my personal PC to Outlook 2007.
mkmurray
Senior Member
#21
boomerbubba wrote:When this application is released, will users have access to your final vCard file (including embedded photos), or will they have to use Outlook? I might want to load the portable .vcf file into some other platform, but I have no plans to upgrade my personal PC to Outlook 2007.
The idea was to make it a two step process with two separate executables, in order to deal with the fact that not everyone will be putting the contacts into Outlook in order to sync with the iPhone. So yes, what you suggest would be possible.

I actually have not worked on this code base in a while. I was starting to think that the iWard and iStake iPhone apps would make mine unnecessary. The only benefit from my code that I can think of was the proposed feature of being able to take MLS vCards and embedding the photos, instead of using the LUWS ones.

I have been paying $5 per month to host a server to distribute the code and provide live updates through the ClickOnce technology. It has been hard to set aside time given the circumstances.

However, I will say it is encouraging that someone even remembered I was working on the project.
RossEvans
Senior Member
#22
mkmurray wrote:I actually have not worked on this code base in a while. I was starting to think that the iWard and iStake iPhone apps would make mine unnecessary.

Only for users who own an iPhone and license a copy of iWard or iStake.

I was thinking about generalized use of the standard vCard file format enhanced with the pictures.
brianpratt
New Member
#23
I am also very interested in this. I am an iPhone user and developer and would love to have the photos from the membership directory on my device. I looked into buying the iWard, etc. apps, but without the photos they aren't much more interesting than just downloading the vcard file from lds.org and syncing it to my phone (to the standard contacts app).

I imagine the iWard folks haven't added this feature due to the same issues you have run into. I would love to have access to your application to create the .vcf with photos for my own use, if you're ready to release it.

I might also be interested in using your source code for a simple iPhone application. Just downloading the .vcf file, doing the matching, and letting the user browse through the contacts by name or photo would be just what I need. If nothing else, it would encourage the other iPhone app developers to add the feature and even more people could have access to it!
RossEvans
Senior Member
#24
brianpratt wrote:I imagine the iWard folks haven't added this feature due to the same issues you have run into. I would love to have access to your application to create the .vcf with photos for my own use, if you're ready to release it.

I'm not an iPhone user, but my impression was that iWard does include the photos. Doesn't it populate them by scraping the LUWS pages?
brianpratt
New Member
#25
boomerbubba wrote:I'm not an iPhone user, but my impression was that iWard does include the photos. Doesn't it populate them by scraping the LUWS pages?
No, unfortunately, the iWard and iStake apps do not include the member photos.
RossEvans
Senior Member
#26
brianpratt wrote:No, unfortunately, the iWard and iStake apps do not include the member photos.

I see. The iWard web site does not seem to show any photos in its demo or marketing pages, so that seems right.

As a non-iHead, I was confused by mkmurray's remark above:
I was starting to think that the iWard and iStake iPhone apps would make mine unnecessary. The only benefit from my code that I can think of was the proposed feature of being able to take MLS vCards and embedding the photos, instead of using the LUWS ones.

I guess such an application would be more generally useful after all. I understand that there are some other factors beyond mkmurray's control that complicate the matter, so I won't nag him.
mkmurray
Senior Member
#27
I'm seeing enough evidence and community encouragement to pick this up again. I'll see if I can finish the project off in the next week or two and get it published and distributed. I'll keep you all posted. Thanks.
RossEvans
Senior Member
#28
mkmurray wrote:I'm seeing enough evidence and community encouragement to pick this up again. I'll see if I can finish the project off in the next week or two and get it published and distributed. I'll keep you all posted. Thanks.

Good. For my own purposes, I still need to solve the problems on the Android to keep me within the four corners of church policy, but I haven't given up on that. (I also would have to persuade our bishopric to call a ward photographer to make even the LUWS photo directory useful to anyone at all in our ward.)

The vCard-with-pictures file seems to be the best generalized solution for export, and your application also would solve the vCard problem for Outlook users.