Akelos Framework v1 forum archive. This forum is no longer maintained. To report bugs please visit https://github.com/akelos/akelos/issues
    • CommentAuthorswninetails
    I am curious and in need of opinions from others who are experienced with akelos.

    I have a system which I need to import data into on a regular basis. This information can be of various model types (notes, users, contacts, etc...).
    Currently the import tries to use the standardizing ability that akelos provides and does a very very very long process of following the architecture one entity at a time, for instance, creating a user record from a webpage which would provide interfaces for phone numbers, address and the like is the exact same process done during a cvs import one record at a time.

    This does not work, the larger the data set the worse the import time and it is not scalable for our needs, for instance it can take up to 8 hours for an import. Being a coder, I want to dive in and create sql statements to achieve the goal but we have many developers adding functionality to the system, each with their own set of business rules, so maintaining import scripts is a huge cause for concern.

    Is there any way to have the business logic and standardization of akelos but have the import power of straight sql?

    Man I hope this makes sense...
    Thanks for any help, suggestions, comments or whatevers... :)
    • CommentAuthordale

    In my experience you essentially get one or the other. You can duplicate your business logic to allow you to speed up your data import or you can use Akelos' models, etc., to maintain your logic and suffer the performance problems related to mass import.

    Some other thoughts:

    • We experienced huge memory problems when importing a large amount of data using Akelos models. Part of it was due to the version of PHP we had that wouldn't clean out circular references very well. Another issue is the number of static references that Akelos keeps. It was difficult to clean all of those out accurately to allow garbage collection.
    • My tests have shown that creating Akelos models can be quite slow (well, slower than you want) due to the fact that it has to create every relationship each time rather than as needed. You may find some performance gains by reducing the number of relationships on your models where possible.
    • Even though model creation is "slow", I've found that most of the performance issues we have are self inflicted. Review your business logic and make sure it's doing what it should be doing, particularly as it relates to database writes. For example, we recently discovered a place where we were looping 30 times and calling ->assign() on a relationship. It turns out assign() performs a save each and every time. In reality all we wanted to do was set an id on a record and then we were able to do a single save at the end. This resulted in cutting the save time of this particular model to a third of what it was.
    • You might find a way to pull your business logic away from the ActiveRecords. In which case, it might be possible to then use ActiveRecords in your day to day use and simplified SQL generation for import. I'm not sure that doing this would ease your burden at all though. The beauty of the ActiveRecord is that it builds the SQL for you.

    If you manage to find a creative solution to this problem I'd be very interested to hear it.