Does WPREALTY automatically import from a RETS server to a WP Post?

[Brian]

I've read through this website and forums extensively but haven't been able to track down the answers to the following technical questions:

1) Using WP REALTY can I setup a CRON job that will grab the feed from my RETS provider, map the XML tags to defined Wordpress custom fields, and automatically import this as a Wordpress post?

1) Along with number 1, if a listing is automatically posted from RETS, then we manually update one of custom fields in Wordpress, what happens if the RETS feed sends over the same listing with updated information? Does all information get overwritten?

3) What format does WP REALTY grab the RETS feed as? Does it save a raw copy of each feed? How are images saved and organized?

4) I see people talking about buying the plugin, becoming a client, and choosing the theme they want to use. I want to build my own theme, buy the plugin, and host it on my own server. Is this even an option with WP REALTY?

Thanks for your help!

[Chad]

Here is what you get from buying this plugin -

First you get the WP Realty plugin (its the same exact core that we start with as well) And you also get access to some commercial add-ons to choose from along with various templates and themes that will be coming very soon.

Next - MLS info: You will need to find out what type of data feed your mls provider offers, they should have rets.

If the have RETS then simply just use the program called VieleRETS (its free).

Our plugin works fine with VieleRETS and we even use it ourselves on some of our smaller market projects. (handles mapping of everything including images etc. and all you have to do is point it to the Open-Realty settings since we are using WP Realty as a fork of OR) ...Keep in mind though that we do not personally support the product nor do we have any "how-to" videos etc.

VieleRETS has their own documentation and there are some forums out there where people share some how-to information - but since every mls is different we wont be trying to offer support for something we don't control.

Now as far as setting up CRON's to work ...well that can be done with anything you want as long as the host provider offers it as a hosted option.

And finally - the part about having this automatically create a post for the mls listings is something that we have already added to our wish list but it would only be set up for featured listings of either the client or his company and is not something that is currently available.

[Brian]

Thanks for the responses Chad!

I have looked into VieleRETS previously and it appears to be the best option for automating the feed from our RETS provider to our server. Based on your response it sounds like I can probably grab the data using VieleRETS and save it as XML or OR format.

From this point I would likely need to create some custom script that would generate a WP post based on XML or OR file.

My next question then is if I do the following what advantage will using the WP Realty plugin provide in this scenario? Assuming that I don't need any listing widgets or the ability to tag listings into my existing posts will WP Realty still be useful? I'm sure the answer is yes but appreciate your help in understanding the benefits.

[Chad]

Well without our plugin how would you integrate the data feed into a embedded wordpress MLS solution?

...If you dont want to take the MLS and embed it then you would be missing the main purpose of having our plugin as a SEO tool to help you to get thousands of listings indexed into google.

Simply dripping the listings into blog post are one thing but its not the best way to get (all of them) into the googles cache because eventually they will go into archive and then you will also be displaying listings that are either sold or no longer on the market...which is opening yourself up to legal issues.

But hey - if you decide to build it that way then come back and drop us a link, I'd love to see it =)

on another note - or plugin has the featured listings already as an rss feed (with thumbnail ) so that could be used to work from as well =)

[Brian]

I am still fairly ignorant in the world of RETS and MLS so I apologize if I am oversimplifying something I don't completely understand.

Here are the components that would make the ideal system I'm looking for:

1) VieleRETS grabs RETS data on a daily basis
2) Another CRON job runs hourly on our server to parse the XML and feed Wordpress with a new post based on the data (since each entry is a standard post in Wordpress it will be as SEO compatible as it gets)
3) Realtor can update the custom fields of any post manually at any time by logging into Wordpress. Also each post could be manually tagged with certain keywords
4) Users could use the standard Wordpress search features to pull listings based on categories or simple search text
5) Finally, another CRON job runs weekly looks for any listings where status = SOLD and unpublishes the post (this would keep the internal record for the realtor but remove it from being further indexed by search engines)

Issues with the above flow would include how to deal with updated or duplicate entries from the RETS feed. This would require time to code around but shouldn't be impossible.

In terms of RSS feeds if you have your listings broken out into different categories you could easily call one category Featured Listings and plug that into a Featured Content Gallery or its own RSS feed.

As far as Google cache goes I would think that any system anywhere could fail to show accurate data if a user knowingly or unknowingly views cached data. So long as the posting is taken down from our server within a timely manner I don't think we would be accountable for what Google has indexed. Having said that the same issue would exist even if I use WP Realty and have embedded an entry on one of my pages, right?

Using the above mentioned process which step would WP Realty fall into that would simplify or enhance the flow?

Again, thank you for your patience and time in clarifying your product :)

[Jared]

Brian, here is my process on nearly all our MLS clients using this very WPRealty plugin;


1) IF and thats IF I use vieleRETS I setup the Source, Target and Extract using the editor and then I manually reduce the files by manually removing fields and or features of each file I have no need for.

2) I download ALL (each and every) MLS update as a CSV file. I never do any other method even if its XML, I still convert it.

3) Using WPRealty, I use a script to generate the listing form elements based on a compressed version of the fields available for all classes. What I mean by that is this;
If I have a RETS update running for Residential, Condo, MultiFam, Land etc, there are times that each class will share many of the same fields yet be named differently.
RESISQFT, LANDSQFT, MULTSQFT, CONDSQFT are all Square Footage so I create one field all will share rather than create each field.

I also join fields as in STREETDIR, STREETNUM, STREETNAME, STREETSUFFIX, STREETPOSTDIR, etc into one field titled Address. I do this with any field or fields that are not going to be used in the search form. I do this for instance on MultiFamily where they have UNIT1BEDS, UNIT2BEDS, UNIT3BEDS, UNIT4BEDS all put into a field for the multifam beds with the labels as part of the data insert.


4) I run a simple script we built that loads CSV data into WPRealty and depending on Office and Agent available in the client site, it will tag listings as "Featured" if the agent is in the WPRealty DB. So, when I run the update daily on a CRON, I filter for the agents MLSID and if it matches with my user in WPRealty it will set that listing to featured, all else are set as NON featured.

5) IF the MLS can provide us with a media server for remote linking the photos, we use a custom addon that is painfully simple which does little more than hotlink the photos. In version 3.0 this will be a regular feature. As for now, we set it up per client request.

6) IF and this is one of those big IF's again. Some MLS providers do not actually allow you to download any status beyond "Active" so as a result we do a dump and reload of listing data on a scheduled basis. IF I can get expired, sold, withdrawn, etc I'll download only those listings gathering only the MLS# so I can run a script that removes them from the WPRealty database on a daily cron, then I run the updates or new listings to keep the DB fresh. When comparing this with full active downloads, we discovered that we experience less than a small percentage of deviation from full data downloads. In my last project the difference was maybe 2 per 10,0000 listings we missed. This is acceptable when dealing with large data sets.

7) IF I have to download images rather than hot linking them, I'll run a script that simply UNLINKS photos matching the MLS# that are no longer active. This way we keep the folders light.

8) In nearly ALL MLS projects we place images in the images/listing_photos/01/ etc and so on. Each class has its own folder and in version 3.0 you have up to 99 folders already established for sorting by ID. This makes for an easier way to manage large photo archives.

Now about your questions / statements

You stated;
3) Realtor can update the custom fields of any post manually at any time by logging into Wordpress. Also each post could be manually tagged with certain keywords.
This is a native feature in WordPress. If however by "custom fields" you mean listing fields, this is NOT a good idea when dealing with MLS empowered sites. We have encountered way to many broken sites as a result of letting agents modify fields.

You stated;
4) Users could use the standard Wordpress search features to pull listings based on categories or simple search text

Searching multiple data sources is possible with clever scripting but Not possible for querying the listings DB with the standard WordPress search. WordPress search is performed against the posts table and since listings are not actually posts you do require a custom search field. Although I have sites using a search field that can be directed to two different "GET" locations, it isn't the ideal for all situations and it would be my suggestion to use a split search form to get the appearance of this feature.

We had once considered the auto generating of posts in the early development of 3.0 when we struggled and built dozens of DB models to come up with a good foundation. We come to realize that after many tests, it would essentially crucify a CMS type web site and proved to be counter productive to good blog sites. The reasons why I'll explain later.

You stated;
5) Finally, another CRON job runs weekly looks for any listings where status = SOLD and unpublishes the post (this would keep the internal record for the realtor but remove it from being further indexed by search engines)

As stated above, we avoided the entire idea after testing, it proved to be counter SEO and counter to good CMS design!
I would not unpublish a post or listing without taking steps to handle the flood of 404's. We had MLS sites in the SOCAL market that literally generated 400 or more error indexing per month. We looked at all our competition before we finally came up with a method for handling the removal of expired, sold and other types of dropped listings. We have a proprietary method we came up with and even blogged about long ago. I'll dig up the post and get you a link.

Jared

[Chad]

Though you're not the first person to conceive the idea of doing this ...and even though you "might" get your listings found on google better by doing this - the fact is that for a really, really small MLS of say less than 4 or 5k listings it could be fine to plan and build out this concept - but on the other hand I know its simply not a good idea at all for anything larger than that because of the table structure of wordpress.

Without having a real mls search that allows clients to quickly drill down to what it is that they are looking for - like they would have using a advanced search form that queries and "displays the data" in a faster and more importantly "Uniform Short Version" method than what the wordpress sql structure can offer...Imagine you did a search and you got back hundreds or ever worse (thousands) of blog post to now try and wade through - :) - not a very pretty thought at ALL -

I feel that the majority of the people will simply move on to a normal site that is offering them all the regular tools that they are comfortable with and accustomed to using like "google map, save search, add to favorites, email when price changes.....on and on ..etc.


Well with that being said - I guess the main thing would be for you to find and hire a programmer to build it the way you want - but keep in mind that there are rules from every data feed provider for how the data is displayed in order to be in compliance etc. (for example this method would never be allowed in Denver )

[Brian]

Chad/Jared,

First, let me thank you for the unbelievable level of detail you put into answering my questions. I had to read your responses a few times but things are starting to sink in.

I agree now that unpublishing posts would be an awful idea. I also understand the performance and scalability issues that you would encounter using a post based system if you went beyond several thousand listings. Though you could paginate the search results to prevent horrid page loading I can still see a lot of server strain trying to grab posts based on WP defined custom fields. You would literally have to index/sort your wordpress posts database constantly just to keep the speed up.

Automatically importing posts into Wordpress is a bad idea. The only reason anyone would probably ever want to use the posts engine as their listing database is if they represent a very small area and are not participating in broker reciprocity.

Having said the above, I think this thread has thoroughly covered the topic and answered my questions. I haven't yet determined if WP Realty is the solution I need, but if this is the level of support you provide then I certainly have 0 hesitations regarding your product. (Don't be too surprised if you see another thread or two pop up from me :) )

Thanks!