emhass: Energy Management for Home Assistant

Related tags

Hardwareemhass
Overview

emhass

EMHASS: Energy Management for Home Assistant

Context

This module was conceived as an energy management optimization tool for residential electric power consumption and production systems. The goal is to optimize the energy use in order to maximize autoconsumption. The main study case is a household where we have solar panels, a grid connection and one or more controllable (deferrable) electrical loads. Including an energy storage system using batteries is also possible in the code. The package is highly configurable with an object oriented modular approach and a main configuration file defined by the user. EMHASS was designed to be integrated with Home Assistant, hence it's name. Installation instructions and example Home Assistant automation configurations are given below.

The main dependencies of this project are PVLib to model power from a PV residential installation and the PuLP Python package to perform the actual optimizations using the Linear Programming approach.

The complete documentation for this package is available here.

Installation

It is recommended to install on a virtual environment. For this you will need virtualenv, install it using:

sudo apt install python3-virtualenv

Then create and activate the virtual environment:

virtualenv -p /usr/bin/python3 emhassenv
cd emhassenv
source bin/activate

Install using the distribution files:

python3 -m pip install emhass

... or the compiled whl file:

pip install emhass-X.X.X-py3-none-any.whl

Clone this repository to obtain the example configuration files. We will suppose that this repository is cloned to:

/home/user/emhass

This will be the root path containing the yaml configuration files (config.yaml and secrets.yaml) and the different needed folders (a data folder to store the optimizations results and a scripts folder containing the bash scripts described further below).

Usage

To run a command simply use the emhass command followed by the needed arguments. The available arguments are:

  • --action: That is used to set the desired action, options are: perfect-optim, dayahead-optim and publish-data
  • --config: Define path to the config.yaml file

For example, the following line command can be used to perform a day-ahead optimization task:

emhass --action 'dayahead-optim' --config '/home/user/emhass'

Before running any valuable command you need to modify the config.yaml and secrets.yaml files.

Home Assistant integration

To integrate with home assistant we will need to define some shell commands in the configuration.yaml file and some basic automations in the automations.yaml file.

In configuration.yaml:

shell_command:
  dayahead_optim: /home/user/emhass/scripts/dayahead_optim.sh
  publish_data: /home/user/emhass/scripts/publish_data.sh

And in automations.yaml:

- alias: EMHASS day-ahead optimization
  trigger:
    platform: time
    at: '05:30:00'
  action:
  - service: shell_command.dayahead_optim
- alias: EMHASS publish data
  trigger:
  - minutes: /5
    platform: time_pattern
  action:
  - service: shell_command.publish_data

In these automations the optimization is performed everyday at 5:30am and the data is published every 5 minutes. Create the file dayahead_optim.sh with the following content:

#!/bin/bash
. /home/user/emhassenv/bin/activate
emhass --action 'dayahead-optim' --config '/home/user/emhass'

And the file publish_data.sh with the following content:

#!/bin/bash
. /home/user/emhassenv/bin/activate
emhass --action 'publish-data' --config '/home/user/emhass'

Then specify user rights and make the files executables:

sudo chmod -R 755 /home/user/emhass/scripts/dayahead_optim.sh
sudo chmod -R 755 /home/user/emhass/scripts/publish_data.sh
sudo chmod +x /home/user/emhass/scripts/dayahead_optim.sh
sudo chmod +x /home/user/emhass/scripts/publish_data.sh

The final action will be to link a sensor value in Home Assistant to control the switch of a desired controllable load. For example imagine that I want to control my water heater and that the publish-data action is publishing the optimized value of a deferrable load that I have linked to my water heater desider behavior. In this case we could use an automation like this one below to control the desired real switch:

automation:
  trigger:
    - platform: numeric_state
      entity_id:
        - sensor.p_deferrable1
      above: 0
  action:
    - service: homeassistant.turn_on
      entity_id: switch.water_heater

Development

To develop using Anaconda:

conda create --name emhass-dev python=3.8 pip=21.0.1

Then activate environment and install emhass using the provided setup.py file:

conda activate emhass-dev
python setup.py install

Add more packages if needed, this is optional if using spyder:

conda install spyder-kernels

Update the build package:

python3 -m pip install --upgrade build

And generate distribution archives with::

python3 -m build

In Spyder you can use CTRL+F6 and add the needed arguments in the "Command line options". The runfile command in the ipython console may look like this:

runfile('/home/user/emhass/src/emhass/command_line.py', args='--action "dayahead-optim" --config "/home/user/emhass"', wdir='/home/user/emhass/src/emhass')

To generate de documentation we will use Sphynx, the following packages are needed:

pip install sphinx==3.5.4 sphinx-rtd-theme==0.5.2 myst-parser==0.14.0

The actual documentation is generated using:

make clean
make html
sphinx-apidoc -o ./ ../src/emhass/

TODO

  • Implement an energy management with a Model Predictive Control approach. Consider implementing the receiding horizon approach.
  • Improve load forecasting using a time series forecast algorithm. Some tests were made with fbprophet but results are not completly satisfactory. The model needs some regressors for more accuracy.
  • Introduce the modeling of constraints during optimization for a thermal energy storage
  • EMHASS hass been tested in Home Assistan Core. It need to be tested on Home Assistant Operating System and Home Assistant Container.
  • Create an EMHASS add-on for even easier installation on Home Assistant Operating System and Home Assistant Supervised
Comments
  • Error passing forecast data

    Error passing forecast data

    When trying to pass forecast data all looks good from the command line:

    [email protected]:~$ curl -i -H "Content-Type: application/json" -X POST -d '{"prod_price_forecast":[0.14, 0.15, 0.27, 0.3, 0.34, 0.29, 0.23, 0.19, 0.16, 0.13, 0.13, 0.12, 0.13, 0.12, 0.15, 0.14, 0.15, 0.14, 0.13, 0.13, 0.13, 0.13, 0.11, 0.11, 0.14, 0.15, 0.15, 0.14, 0.22, 0.37, 0.37, 0.37, 0.37, 0.3, 0.37, 0.37, 0.16, 0.15, 0.15, 0.15, 0.16, 0.22, 0.37, 0.37, 0.37, 0.37, 0.37, 0.37]}' http://localhost:5000/action/dayahead-optim
    HTTP/1.1 201 CREATED
    Server: Werkzeug/2.1.1 Python/3.9.2
    Date: Mon, 25 Apr 2022 05:45:55 GMT
    Content-Type: text/html; charset=utf-8
    Content-Length: 45
    
    EMHASS >> Action dayahead-optim executed... 
    [email protected]:~$ curl -i -H "Content-Type: application/json" -X POST -d '{"load_cost_forecast":[0.25, 0.26, 0.39, 0.42, 0.46, 0.42, 0.35, 0.31, 0.27, 0.24, 0.24, 0.23, 0.24, 0.23, 0.26, 0.26, 0.26, 0.25, 0.24, 0.24, 0.24, 0.24, 0.22, 0.21, 0.25, 0.26, 0.26, 0.25, 0.34, 0.5, 0.51, 0.51, 0.5, 0.42, 0.51, 0.5, 0.28, 0.27, 0.26, 0.26, 0.27, 0.34, 0.5, 0.51, 0.51, 0.51, 0.51, 0.51]}' http://localhost:5000/action/dayahead-optim
    HTTP/1.1 201 CREATED
    Server: Werkzeug/2.1.1 Python/3.9.2
    Date: Mon, 25 Apr 2022 05:46:33 GMT
    Content-Type: text/html; charset=utf-8
    Content-Length: 45
    
    EMHASS >> Action dayahead-optim executed... 
    [email protected]:~$ curl -i -H "Content-Type: application/json" -X POST -d '{"pv_power_forecast":[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 70, 141.22, 246.18, 513.5, 753.27, 1049.89, 1797.93, 1697.3, 3078.93, 1164.33, 1046.68, 1559.1, 2091.26, 1556.76, 1166.73, 1516.63, 1391.13, 1720.13, 820.75, 804.41, 251.63, 79.25, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}' http://localhost:5000/action/dayahead-optim
    HTTP/1.1 201 CREATED
    Server: Werkzeug/2.1.1 Python/3.9.2
    Date: Mon, 25 Apr 2022 05:47:30 GMT
    Content-Type: text/html; charset=utf-8
    Content-Length: 45
    
    EMHASS >> Action dayahead-optim executed... 
    [email protected]:~$ 
    

    However it doesn't get updated in the model and the logs complain about length needing to be 48, but then confirming length is 48.

    [2022-04-25 15:45:40,694] ERROR in app_server: ERROR: The passed data is either not a list or the length is not correct, length should be 48
    [2022-04-25 15:45:40,695] ERROR in app_server: Passed type is <class 'list'> and length is 48
    [2022-04-25 15:45:55,225] WARNING in optimization: Failed LP solve with PULP_CBC_CMD solver, falling back to default Pulp
    [2022-04-25 15:45:55,252] WARNING in optimization: Failed LP solve with default Pulp solver, falling back to GLPK_CMD
    172.30.32.1 - - [25/Apr/2022 15:45:55] "POST /action/dayahead-optim HTTP/1.1" 201 -
    [2022-04-25 15:46:18,650] ERROR in app_server: ERROR: The passed data is either not a list or the length is not correct, length should be 48
    [2022-04-25 15:46:18,653] ERROR in app_server: Passed type is <class 'list'> and length is 48
    [2022-04-25 15:46:33,131] WARNING in optimization: Failed LP solve with PULP_CBC_CMD solver, falling back to default Pulp
    [2022-04-25 15:46:33,155] WARNING in optimization: Failed LP solve with default Pulp solver, falling back to GLPK_CMD
    172.30.32.1 - - [25/Apr/2022 15:46:33] "POST /action/dayahead-optim HTTP/1.1" 201 -
    [2022-04-25 15:47:14,370] ERROR in app_server: ERROR: The passed data is either not a list or the length is not correct, length should be 48
    [2022-04-25 15:47:14,373] ERROR in app_server: Passed type is <class 'list'> and length is 48
    [2022-04-25 15:47:30,050] WARNING in optimization: Failed LP solve with PULP_CBC_CMD solver, falling back to default Pulp
    [2022-04-25 15:47:30,083] WARNING in optimization: Failed LP solve with default Pulp solver, falling back to GLPK_CMD
    172.30.32.1 - - [25/Apr/2022 15:47:30] "POST /action/dayahead-optim HTTP/1.1" 201 -
    
    opened by purcell-lab 29
  • optimisation commands not updating data

    optimisation commands not updating data

    Around 26th August, the optimisation process stopped running on my HAOS install.

    I run 2 calls, 5.47am Day Ahead, and regularly throughout day the perfect with external solar, price and battery state being sent. its worked fine for 6 months.

    when I goto webUI:5000 I run the commands and observe the charts do not update.

    I have uninstalled, and reinstalled, and same data set is maintained.

    I think the DB is correct in EMHASS. How can I clear this?

    Today is 28 August, this shows 26 aug: image

    opened by kcoffau 12
  • FR: Return forecast values to home assistant sensor entity attributes

    FR: Return forecast values to home assistant sensor entity attributes

    A feature request.

    The forecast values are currently written to the .csv file within the file system, but are unable to be accessed by home assistant.

    Would it be possible to include the forecasts as attributes of the sensor entities published to home assistant.

    This would enable the creation of the same chart of values as the web UI, but in Lovelace using the apexcharts-card

    For example the SolCast integration posts forecast PV through a series of sensors and attributes. image

    My energy provider also provides future price forecasts in entity attributes: image

    When visualized in apexcharts looks like this: image

    opened by purcell-lab 11
  • Feature request: Make own forecast data persistent

    Feature request: Make own forecast data persistent

    After the load_cost_forecast and prod_price_forecast from Nordpool addon are passed as a list to emhass the data is no persistent. When restarting emhass the data is lost and the default values are used in unit_load_cast and unit_prod_price.

    This cause problems: When I use the service: Shell Command: post_nordpool_forecast to update the data, the data do not exist because Nordpool are only updated 13:00 and are only updated once from Nordpool. or The data exist, but today and tomorrow data from post_nordpool_forecast are not correct placed in time Example: Tomorrow price data - 0.351 are placed in 21:00-22:00, but the correct time is 00:00-01:00.

    It would have been nice if the forecast data could have been stored in emhass and not lost when emhass are restarted.

    opened by overas 9
  • Passing forecast data do not work

    Passing forecast data do not work

    I have posted most of the description of the problem here: https://community.home-assistant.io/t/emhass-an-energy-management-for-home-assistant/338126/111?u=haraldov

    What is the Home Assistant log saying when you execute that shell command?

    2022-08-28 20:38:03.701 DEBUG (MainThread) [homeassistant.components.shell_command] Stdout of command: curl -i -H "Content-Type: application/json" -X POST -d '{"load_cost_forecast": {{ ((( state_attr('sensor.nordpool', 'raw_today') + state_attr('sensor.nordpool', 'raw_tomorrow')) |map(attribute='value')|list)[:48]) }}' http://localhost:5000/action/dayahead-optim, return code: 0: 2022-08-28 20:38:03.701 DEBUG (MainThread) [homeassistant.components.shell_command] Stderr of command: curl -i -H "Content-Type: application/json" -X POST -d '{"load_cost_forecast": {{ ((( state_attr('sensor.nordpool', 'raw_today') + state_attr('sensor.nordpool', 'raw_tomorrow')) |map(attribute='value')|list)[:48]) }}' http://localhost:5000/action/dayahead-optim, return code: 0: ➜ config

    However it doesn't get updated in the model.

    Home Assistant 2022.8.7 Supervisor 2022.08.3 Operating System 8.5 Frontend 20220802.0 - latest Raspberry Pi 4

    EMHASS Current version: 0.2.20

    [2022-08-28 00:14:04,984] INFO in web_server: Launching the emhass webserver at: http://0.0.0.0:5000 [2022-08-28 00:14:04,985] INFO in web_server: Home Assistant data fetch will be performed using url: http://supervisor/core/api [2022-08-28 00:14:04,985] INFO in web_server: The base path is: /usr/src [2022-08-28 00:14:04,989] INFO in web_server: Using core emhass version: 0.3.18 [2022-08-28 00:15:24,545] INFO in web_server: EMHASS server online, serving index.html... [2022-08-28 00:20:29,186] INFO in web_server: EMHASS server online, serving index.html... [2022-08-28 00:22:31,750] INFO in web_server: EMHASS server online, serving index.html... [2022-08-28 00:22:33,664] INFO in command_line: Setting up needed data [2022-08-28 00:22:33,759] INFO in forecast: Retrieving weather forecast data using method = scrapper [2022-08-28 00:22:39,605] INFO in forecast: Retrieving data from hass for load forecast using method = naive [2022-08-28 00:22:39,607] INFO in retrieve_hass: Retrieve hass get data method initiated... [2022-08-28 00:22:53,887] INFO in web_server: EMHASS server online, serving index.html... [2022-08-28 00:22:56,188] INFO in web_server: >> Performing dayahead optimization... [2022-08-28 00:22:56,189] INFO in command_line: Performing day-ahead forecast optimization [2022-08-28 00:22:56,198] INFO in optimization: Perform optimization for the day-ahead [2022-08-28 00:22:56,755] INFO in optimization: Status: Optimal [2022-08-28 00:22:56,756] INFO in optimization: Total value of the Cost function = -2.02 [2022-08-28 20:37:38,794] INFO in web_server: EMHASS server online, serving index.html... [2022-08-28 20:38:27,230] INFO in web_server: EMHASS server online, serving index.html...

    System Information

    version | core-2022.8.7 -- | -- installation_type | Home Assistant OS dev | false hassio | true docker | true user | root virtualenv | false python_version | 3.10.5 os_name | Linux os_version | 5.15.32-v8 arch | aarch64 timezone | Europe/Oslo config_dir | /config

    Home Assistant Community Store

    GitHub API | ok -- | -- GitHub Content | ok GitHub Web | ok GitHub API Calls Remaining | 5000 Installed Version | 1.26.2 Stage | running Available Repositories | 1164 Downloaded Repositories | 12

    Home Assistant Cloud

    logged_in | true -- | -- subscription_expiration | September 7, 2022 at 02:00 relayer_connected | true remote_enabled | true remote_connected | true alexa_enabled | false google_enabled | true remote_server | eu-west-2-3.ui.nabu.casa can_reach_cert_server | ok can_reach_cloud_auth | ok can_reach_cloud | ok

    Easee EV Charger

    component_version | 0.9.44 -- | -- reach_easee_cloud | ok connected2stream | true

    Home Assistant Supervisor

    host_os | Home Assistant OS 8.5 -- | -- update_channel | stable supervisor_version | supervisor-2022.08.3 agent_version | 1.2.1 docker_version | 20.10.14 disk_total | 118.3 GB disk_used | 14.8 GB healthy | true supported | true board | rpi4-64 supervisor_api | ok version_api | ok installed_addons | ESPHome (2022.3.1), SSH & Web Terminal (12.0.2), Mosquitto broker (6.1.2), File editor (5.3.3), Frigate NVR Proxy (1.3), Home Assistant Google Drive Backup (0.108.4), Z-Wave JS to MQTT (0.45.0), EMHASS (0.2.20)

    Dashboards

    dashboards | 1 -- | -- resources | 2 views | 10 mode | storage

    Recorder

    oldest_recorder_run | August 25, 2022 at 08:33 -- | -- current_recorder_run | August 28, 2022 at 09:14 estimated_db_size | 575.15 MiB database_engine | sqlite database_version | 3.38.5

    Spotify

    api_endpoint_reachable | ok -- | --

    opened by overas 9
  • Solcast 2 rooftop ids

    Solcast 2 rooftop ids

    I have 2 solcast rooftop ids, how can i pass that data? I'm also using https://github.com/oziee/ha-solcast-solar, so I have solcast sensors in my HA, is it possible to pass that data?

    opened by gieljnssns 7
  • datetime.now() method consequences for submitted forecasts

    datetime.now() method consequences for submitted forecasts

    The publish-data command will load the latest csv file and look for the closest timestamp that match the current time using the datetime.now() method in Python. This means that if EMHASS is configured for 30min time step optimizations, the csv will be saved with timestamps 00:00, 00:30, 01:00, 01:30, … and so on. If the current time is 00:05, then the closest timestamp of the optimization results that will be published is 00:00. If the current time is 00:25, then the closest timestamp of the optimization results that will be published is 00:30.

    One (unintended) consequence of the datetime.now() method results when submitting forecast data.

    Consider the data from my energy provider: image

    Which is represented in HA as the following entities and attributes:

    sensor.amber_general_price: 0.36
    sensor.amber_general_forecast: 0.37
    'forecasts.per_kwh': [0.37, 0.38, 0.38, 0.38, 0.38, 0.38, 0.38, 0.38, 0.39, 0.43, 0.45, 0.72, 0.73, 0.73, 0.72, 0.45, 0.7, 0.46, 0.45, 0.39, 0.36, 0.32, 0.32, 0.33, 0.7, 0.7, 0.51, 0.43, 0.51, 0.61, 0.46, 0.47, 0.61, 15.12, 15.08, 0.74, 0.52, 0.61, 0.52, 0.45, 0.44, 0.43, 0.45, 0.44, 0.38, 0.36, 0.43, 0.39]
    
    • If I conduct an optimisation at 00:35 the optimisation results will be published for 00:30, so the first value needs to be the now value, followed by the forecast for 01:00, then 01:30 and so on.
    "load_cost_forecast":(sensor.amber_general_price + forecasts.per_kwh)[:48]
    "load_cost_forecast": [0.36, 0.37, 0.38, 0.38, ...]
    
    [:48] is to truncate to 48 elements as that list will have 49 elements
    
    • If I conduct an optimisation at 00:55 the optimisation results will be published for 01:00, so the first value needs to be the forecast value for 01:00, the 01:30 and so on. The Now value isn't used.
    "load_cost_forecast":(forecasts.per_kwh)
    "load_cost_forecast":[0.37, 0.38, 0.38, 0.38, ...]
    

    Whilst not impossible to calculate the changing load_cost_forecast, it is confusing as the data it requires changes depending on which 15 minute block it is generated in.

    Additionally I would submit the first optimisation including the now value is superior as it includes more data, to base the optimisation on and this is also important for the MPC as it will be called with high frequency (maybe every 5 minutes).

    Could I propose that the optimisation should always include the now value as the first element of the forecast list and that for consistency the published optimisation shouldn't map to the closest timestamp (which maybe ahead of behind), but that the published optimisation is always published with the start of the current time step. So if I publish at either 00:05 or 00:25 it will be published for 00:00, if I publish at either 00:35 or 00:55 it will be published for 00:30.

    opened by purcell-lab 5
  • Multiple deferrable_loads

    Multiple deferrable_loads

    I wan't to use multiple deferrable_loas, like dishwasher, washing machine, dryer, hot water, boost of my thermostat Some of my deferrable_loads aren't needed every day. Is there a way to tell to emhass if that deferrable_load is needed? Maybe an input_boolean to controle every deferrable_load?

    opened by gieljnssns 4
  • FR: Pass additional sensor entities.

    FR: Pass additional sensor entities.

    Working quite well here now and stable.

    Would it be possible to pass some additional sensor entities to HA?

    p_grid_forecast & total value of cost function

    Thanks.

    My optimisation for next 24 hrs, total value of cost function $63:

    image

    enhancement 
    opened by purcell-lab 4
  • KeyError: 'P_batt'

    KeyError: 'P_batt'

    Running 0.1.37 and feels like we are getting close :-).

    When I enable the battery for the solution space dayahead doesn't seem to finish, running for over 10 minutes before I called again:

    [s6-init] making user provided files available at /var/run/s6/etc...exited 0.
    [s6-init] ensuring user provided files have correct perms...exited 0.
    [fix-attrs.d] applying ownership & permissions fixes...
    [fix-attrs.d] done.
    [cont-init.d] executing container initialization scripts...
    [cont-init.d] done.
    [services.d] starting services
    [services.d] done.
     * Serving Flask app 'app_server' (lazy loading)
     * Environment: production
       WARNING: This is a development server. Do not use it in a production deployment.
       Use a production WSGI server instead.
     * Debug mode: off
     * Running on all addresses (0.0.0.0)
       WARNING: This is a development server. Do not use it in a production deployment.
     * Running on http://127.0.0.1:5000
     * Running on http://172.30.33.4:5000 (Press CTRL+C to quit)
    [2022-04-28 13:27:11,810] WARNING in optimization: Failed LP solve with PULP_CBC_CMD solver, falling back to default Pulp
    [2022-04-28 13:27:11,866] WARNING in optimization: Failed LP solve with default Pulp solver, falling back to GLPK_CMD
    192.168.86.50 - - [28/Apr/2022 13:38:18] "GET / HTTP/1.1" 200 -
    192.168.86.50 - - [28/Apr/2022 13:38:18] "GET /static/style.css HTTP/1.1" 304 -
    [2022-04-28 13:45:39,692] WARNING in optimization: Failed LP solve with PULP_CBC_CMD solver, falling back to default Pulp
    [2022-04-28 13:45:39,742] WARNING in optimization: Failed LP solve with default Pulp solver, falling back to GLPK_CMD
    

    Then the next time publish runs it fails:

    [s6-init] making user provided files available at /var/run/s6/etc...exited 0.
    [s6-init] ensuring user provided files have correct perms...exited 0.
    [fix-attrs.d] applying ownership & permissions fixes...
    [fix-attrs.d] done.
    [cont-init.d] executing container initialization scripts...
    [cont-init.d] done.
    [services.d] starting services
    [services.d] done.
     * Serving Flask app 'app_server' (lazy loading)
     * Environment: production
       WARNING: This is a development server. Do not use it in a production deployment.
       Use a production WSGI server instead.
     * Debug mode: off
     * Running on all addresses (0.0.0.0)
       WARNING: This is a development server. Do not use it in a production deployment.
     * Running on http://127.0.0.1:5000
     * Running on http://172.30.33.4:5000 (Press CTRL+C to quit)
    192.168.86.50 - - [28/Apr/2022 13:15:43] "GET / HTTP/1.1" 200 -
    192.168.86.50 - - [28/Apr/2022 13:15:44] "GET /static/style.css HTTP/1.1" 304 -
    [2022-04-28 13:16:10,105] WARNING in optimization: Failed LP solve with PULP_CBC_CMD solver, falling back to default Pulp
    [2022-04-28 13:16:10,160] WARNING in optimization: Failed LP solve with default Pulp solver, falling back to GLPK_CMD
    [2022-04-28 13:20:17,487] ERROR in app: Exception on /action/publish-data [POST]
    Traceback (most recent call last):
      File "/usr/local/lib/python3.9/dist-packages/pandas/core/indexes/base.py", line 3621, in get_loc
        return self._engine.get_loc(casted_key)
      File "pandas/_libs/index.pyx", line 136, in pandas._libs.index.IndexEngine.get_loc
      File "pandas/_libs/index.pyx", line 163, in pandas._libs.index.IndexEngine.get_loc
      File "pandas/_libs/hashtable_class_helper.pxi", line 5198, in pandas._libs.hashtable.PyObjectHashTable.get_item
      File "pandas/_libs/hashtable_class_helper.pxi", line 5206, in pandas._libs.hashtable.PyObjectHashTable.get_item
    KeyError: 'P_batt'
    The above exception was the direct cause of the following exception:
    Traceback (most recent call last):
      File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 2077, in wsgi_app
        response = self.full_dispatch_request()
      File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1525, in full_dispatch_request
        rv = self.handle_user_exception(e)
      File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1523, in full_dispatch_request
        rv = self.dispatch_request()
      File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1509, in dispatch_request
        return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
      File "/usr/src/app_server.py", line 191, in action_call
        _ = publish_data(input_data_dict, app.logger)
      File "/usr/local/lib/python3.9/dist-packages/emhass/command_line.py", line 182, in publish_data
        input_data_dict['rh'].post_data(opt_res_dayahead['P_batt'], idx_closest,
      File "/usr/local/lib/python3.9/dist-packages/pandas/core/frame.py", line 3505, in __getitem__
        indexer = self.columns.get_loc(key)
      File "/usr/local/lib/python3.9/dist-packages/pandas/core/indexes/base.py", line 3623, in get_loc
        raise KeyError(key) from err
    KeyError: 'P_batt'
    172.30.32.1 - - [28/Apr/2022 13:20:17] "POST /action/publish-data HTTP/1.1" 500 -
    
    web_ui_url: 0.0.0.0
    hass_url: empty
    long_lived_token: empty
    costfun: profit
    optimization_time_step: 30
    historic_days_to_retrieve: 2
    sensor_power_photovoltaics: sensor.apf_generation_entity
    sensor_power_load_no_var_loads: sensor.power_load_no_var_loads
    number_of_deferrable_loads: 2
    nominal_power_of_deferrable_loads: 5000,1500
    operating_hours_of_each_deferrable_load: 5,8
    peak_hours_periods_start_hours: 02:54,17:24
    peak_hours_periods_end_hours: 15:24,20:24
    load_peak_hours_cost: 0.1907
    load_offpeak_hours_cost: 0.1419
    photovoltaic_production_sell_price: 0.065
    maximum_power_from_grid: 30000
    pv_module_model: CSUN_Eurasia_Energy_Systems_Industry_and_Trade_CSUN295_60M
    pv_inverter_model: Fronius_International_GmbH__Fronius_Primo_5_0_1_208_240__240V_
    surface_tilt: 30
    surface_azimuth: 205
    modules_per_string: 16
    strings_per_inverter: 1
    set_use_battery: true
    battery_discharge_power_max: 5000
    battery_charge_power_max: 5000
    battery_discharge_efficiency: 0.95
    battery_charge_efficiency: 0.95
    battery_nominal_energy_capacity: 13500
    battery_minimum_state_of_charge: 0.05
    battery_maximum_state_of_charge: 1
    battery_target_state_of_charge: 0.2
    
    opened by purcell-lab 4
  • Update docutils requirement from <0.18 to <0.20

    Update docutils requirement from <0.18 to <0.20

    Updates the requirements on docutils to permit the latest version.

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 3
  • Feature request: Access EMHASS web interface via ingress

    Feature request: Access EMHASS web interface via ingress

    Ingress provides a means for an Home Assistant addon to expose its web service via the HA UI:

    https://www.home-assistant.io/blog/2019/04/15/hassio-ingress/ https://developers.home-assistant.io/docs/add-ons/presentation/#ingress

    Currently the EMHASS Web User Interface is accessed via http://homeassistant-ip:5000.

    It would greatly simplify the user experience if EMHASS was configured to utilise the ingress interface.

    My other add-on's are access via the Home Assistant UI and ingress interface:

    ESPHome: http://odroid.local:8123/hassio/ingress/5c53de3b_esphome Google Drive Backup: http://odroid.local:8123/hassio/ingress/cebe7a76_hassio_google_drive_backup Terminal & SSH: http://odroid.local:8123/hassio/ingress/core_ssh

    image image

    enhancement help wanted 
    opened by purcell-lab 2
Releases(v0.3.21)
  • v0.3.21(Oct 21, 2022)

    Fix

    • Fixed docstrings
    • Added github worflows for coverage and automatic package compiling
    • Fixing interpolation for Forecast.Solar data

    Full Changelog: https://github.com/davidusb-geek/emhass/compare/v0.3.20...v0.3.21

    Source code(tar.gz)
    Source code(zip)
  • v0.3.20(Oct 5, 2022)

    A new release with bunch of improvements, most notably added direct support for the Solar.Forecast method to forecast PV power.

    Also:

    • Added more detailed examples to the forecast module documentation.
    • Improved handling of datatime indexes in DataFrames on forecast module.
    • Added warning messages if passed list values contains non numeric items.
    • Added missing unittests for forecast module with request.get dependencies using MagicMock.

    What's Changed

    • Dev/mockgetrequest by @davidusb-geek in https://github.com/davidusb-geek/emhass/pull/25

    Full Changelog: https://github.com/davidusb-geek/emhass/compare/v0.3.19...v0.3.20

    Source code(tar.gz)
    Source code(zip)
  • v0.3.19(Sep 14, 2022)

    Fix

    • Updated default values for a working LP solver.
    • Removed option to provide a custom web ui url.
    • Added extra runtime parameters to use solcast PV forecast.

    Full Changelog: https://github.com/davidusb-geek/emhass/compare/v0.3.18...v0.3.19

    Source code(tar.gz)
    Source code(zip)
  • v0.3.18(Aug 27, 2022)

    Improvement

    • Improving documentation, added more information on forecast page.
    • Added support for SolCast PV production forecasts.
    • Added possibility to pass some optimization parameters at runtime.
    • Added some unittest for passing data as list testing.

    Fix

    • Fixed small bug on webserver using pandas sum function for non numeric data. This was throwing the following message: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated.

    Full Changelog: https://github.com/davidusb-geek/emhass/compare/v0.3.17...v0.3.18

    Source code(tar.gz)
    Source code(zip)
  • v0.3.17(Jun 12, 2022)

    Some bug fixes.

    Fix

    • Fixed wrong variables names for mixed forecasts.
    • Fixed handling of load sensor name in command line setup function.

    Full Changelog: https://github.com/davidusb-geek/emhass/compare/v0.3.16...v0.3.17

    Source code(tar.gz)
    Source code(zip)
  • v0.3.16(Jun 10, 2022)

    Some worthy improvements in this new version.

    Improvement

    • Improving documentation, added "what is this" section and added some info-graphics.
    • Added new forecasts methods chapter in documentation.
    • Added publish of sensors for p_grid_forecast & total value of cost function.
    • Implemented now/current value forecast correction when using MPC.

    Full Changelog: https://github.com/davidusb-geek/emhass/compare/v0.3.15...v0.3.16

    Source code(tar.gz)
    Source code(zip)
  • v0.3.15(Jun 9, 2022)

    Fix

    • Fixed small bug with wrong DF variable name in web server.

    Full Changelog: https://github.com/davidusb-geek/emhass/compare/v0.3.14...v0.3.15

    Source code(tar.gz)
    Source code(zip)
  • v0.3.14(Jun 5, 2022)

    Improvement

    • Added one more table to the weu showing the cost totals.

    Fix

    • Fixed wrong type error when serializing numpy ints. Converted ints to Python type.

    Full Changelog: https://github.com/davidusb-geek/emhass/compare/v0.3.13...v0.3.14

    Source code(tar.gz)
    Source code(zip)
  • v0.3.13(May 30, 2022)

    Fix

    • Fix wrong default value implementation for solver params.

    Improvement

    • Added support to provide solver name and path as parameters in the configuration file.

    What's Changed

    • Update yaxis title for fig 2. by @purcell-lab in https://github.com/davidusb-geek/emhass/pull/10

    Full Changelog: https://github.com/davidusb-geek/emhass/compare/v0.3.11...v0.3.13

    Source code(tar.gz)
    Source code(zip)
  • v0.3.11(May 23, 2022)

    In this new version some improvements were added to the code.

    Improvement

    • Added data as attributes for forecasts (PV and load), deferrable loads and battery: power+SOC.
    • Improved the graph in the webserver, now using subplots.
    • Rearranged vertical space in index.html template.

    Fix

    • Added threads option on waitress to possibly improve queue problem.

    Full Changelog: https://github.com/davidusb-geek/emhass/compare/v0.3.10...v0.3.11

    Source code(tar.gz)
    Source code(zip)
  • v0.3.9(May 19, 2022)

    Several bug fixes and improvements.

    Improvement

    • Improved publish data logging
    • Round published data
    • Attributes to forecasts published data
    • Improved centering html on small devices
    • Improved handling of closest index when publishing data

    Fix

    • Fixed problem with csv filenames, was using only filename specific to dayahead optimization.
    • Fixed data list handling for load cost and prod price forecasts.
    • Fixed publish data dictionary to contain only data from correct entity_id.
    • May have solved double log lines.

    Full Changelog: https://github.com/davidusb-geek/emhass/compare/v0.3.8...v0.3.9

    Source code(tar.gz)
    Source code(zip)
  • v0.3.8(May 17, 2022)

    Fix

    • Still fixing issues when passing csv files and lists.
    • Fixed None weather df issue when passing lists.
    • Improved command line unittests.

    Full Changelog: https://github.com/davidusb-geek/emhass/compare/v0.3.7...v0.3.8

    Source code(tar.gz)
    Source code(zip)
  • v0.3.6(May 17, 2022)

  • v0.3.0(May 11, 2022)

    A lot of improvements in this new release of EMHASS. Most notably the integration of a new Model Predictive Controller (MPC) !!! But also added a new production webserver and a new Dockerfile for a standalone docker mode.

    Improvement

    • Improved the command line setup function to perform the correct amount calls as needed by each action.
    • Added a new naive model predictive control function.
    • Added runtime parameter option for better code order.
    • Moved treatment of runtime parameters from the add-on to the core emhass module. This adds more clarity to the code andd also was needed when passing runtime paramters to emhass in standalone mode.
    • Added add-on paramter to command line to define if launching emhass from add-on or in standalone mode.
    • Added new testing file for command_line.
    • Added a webserver. Moved the webserver from the add-on to the core emhass module.
    • Added a WSGI production server for flask using waitress.
    • Added a Dockerfile and procedure to run emhass in standalone mode.
    • Updated the documentation.

    Full Changelog: https://github.com/davidusb-geek/emhass/compare/v0.2.14...v0.3.0

    Source code(tar.gz)
    Source code(zip)
  • v0.2.14(May 5, 2022)

    Improvement

    • Added more info on publish data errors when not key found. This error may mean that the optimization task may need to be relaunched or it did not converged to a solution.
    • Added better info to the configuration documentation when integrating PV module and inverter models from PVLib database. An underscore _ character should be added inplace of each special character.

    Fix

    • Fixed missing timeStep product for correct deferrable load total energies.

    Full Changelog: https://github.com/davidusb-geek/emhass/compare/v0.2.13...v0.2.14

    Source code(tar.gz)
    Source code(zip)
  • v0.2.13(May 1, 2022)

    Improvement

    • Added support to pass list of PV plant parameters. This will enable to simulate mixed orientation systems, for example one east-facing array (azimuth=90) and one west-facing array (azimuth=270).

    Fix

    • Fixed issue computing correct final cost values.

    Full Changelog: https://github.com/davidusb-geek/emhass/compare/v0.2.12...v0.2.13

    Source code(tar.gz)
    Source code(zip)
  • v0.2.12(Apr 29, 2022)

    This release adds a new parameter to allow consider two different cases for self-consumption:

    1. The case where all PV produced power is injected to the grid, with no direct self-consumption. This case is very popular in France an maybe in many other places. House that are in this case have two energy counters, one for producer energy and another one for consumed energy.

    2. The case of direct self-consumption. The house load can be directly provided by the PV production. This may be a more generalize worldwide case for PV production.

    Improvement

    • Added config parameter to consider that all PV power is injected to the grid

    Full Changelog: https://github.com/davidusb-geek/emhass/compare/v0.2.11...v0.2.12

    Source code(tar.gz)
    Source code(zip)
  • v0.2.11(Apr 28, 2022)

    Fixed problem handling the index of passed data as list of values. This was also affecting reading from a csv file. Also improved unittests to test this kind of issues.

    Fix

    • Fixed wrong handling of DateTimeIndex when dealing with forecast method for list of values and csv read.

    Full Changelog: https://github.com/davidusb-geek/emhass/compare/v0.2.10...v0.2.11

    Source code(tar.gz)
    Source code(zip)
  • v0.2.10(Apr 26, 2022)

    There was a wrong handling of dataframe indexes when importing forecasts using list of values and csv files.

    Fix

    • Fixed faulty forecast method for list of values and csv read.

    What's Changed

    • Minor spelling updates by @purcell-lab in https://github.com/davidusb-geek/emhass/pull/3
    • Pass only 48 data items in list by @purcell-lab in https://github.com/davidusb-geek/emhass/pull/2

    New Contributors

    • @purcell-lab made their first contribution in https://github.com/davidusb-geek/emhass/pull/3

    Full Changelog: https://github.com/davidusb-geek/emhass/compare/v0.2.9...v0.2.10

    Source code(tar.gz)
    Source code(zip)
  • v0.2.9(Apr 21, 2022)

    Releasing this version to update the get_loc pandas method previously used as it is now deprecated.

    Fix

    • Fixed get_loc deprecation warning using now get_indexer pandas method. Improved selection of closest index.
    Source code(tar.gz)
    Source code(zip)
  • v0.2.8(Apr 18, 2022)

    Again two sort of fatal errors fixed. These were related to publish_data. Tested and working fine with add-on on dev-container.

    Fix

    • Fixed if sentence to correctly use the supervisor API for publish data.
    • Fixing a error computing the nearest index of DataFrame. Using a try/catch strategy to use nearest method as a backup.
    Source code(tar.gz)
    Source code(zip)
  • v0.2.7(Apr 18, 2022)

    This release is basically fixing a fatal error where the publish data function will never reach the saved csv file as the default filename is not equal to the expected filename in publish_data.

    Source code(tar.gz)
    Source code(zip)
  • v0.2.6(Apr 16, 2022)

    This release features a better handling of errors when trying to find a suitable solver.

    Some problems may arise from solver related issues in the Pulp package. It was found that for arm64 architectures (ie. Raspberry Pi4, 64 bits) the default solver is not avaliable. A workaround is to install a new solver. The glpk solver is an option and can be installed with:

    sudo apt-get install glpk-utils
    

    After this it should be available for use and EMHASS can use it as a fallback option.

    Improvement

    • Improved handling of errors concerning solver issues with Pulp. Added support for glpk solver. For now just using a try/catch strategy but should update to solver passed as a parameter to EMHASS.
    Source code(tar.gz)
    Source code(zip)
  • v0.2.5(Apr 12, 2022)

    The previous release didn't last long due to a fatal missing import in utils.

    Fix

    • Fixed missing numpy import in utils.

    Full Changelog: https://github.com/davidusb-geek/emhass/compare/v0.2.4...v0.2.5

    Source code(tar.gz)
    Source code(zip)
  • v0.2.4(Apr 12, 2022)

    Some minor but important changes in this release. Most notably the handling of freq definition to DataFrame when retrieving data from Home Assistant.

    Fix

    • Fixed missing command to retrieve PV power forecast when using list of values.
    • Updated handling of freq definition to a pandas index.

    Full Changelog: https://github.com/davidusb-geek/emhass/compare/v0.2.3...v0.2.4

    Source code(tar.gz)
    Source code(zip)
  • v0.2.3(Apr 2, 2022)

    In this release support for passing list of values to the different forecasting methods was added. This will allow users of the add-on to easily transfer their own forecasting values from other Home Assistant services. Another important change is the dropout of any forecasting method from PVLib, as these methods were marked as deprecated in the latest version of PVLib. Of course PVLib will stll be used to convert weather data to electrical power. Finally the documentation was updated.

    Added

    • Improved support for the new add-on and direct communication via the supervisor.
    • The CLI now can return the version using the --version argument.
    • Improved comments in forecast class.
    • Added unittest for csv method for weather forecast.
    • Added support for passing lists of values to all forecasting methods.

    Fix

    • Removed dependency from PVLib Forecast class, as it has been marked as deprecated.
    • Fixed failing docs builds due to uncompatible jinja2 an markupsafe versions.

    Full Changelog: https://github.com/davidusb-geek/emhass/compare/v0.2.2...v0.2.3

    Source code(tar.gz)
    Source code(zip)
  • v0.2.2(Mar 5, 2022)

    This release is marked with two main changes:

    1. Adding CI workflow for better coding best practices
    2. Changed the handling and naming conventions for the configuration files of the module. These changes are meant to pave the road for future development of HASS add-ons

    Added

    • Added a new input data file using pickle for tests.
    • Added support to select if the logger should save to a file or not.
    • Added CI workflow using github actions.

    Breaking changes

    • Changed package usage of configuration file path, now the user must provide the complete path including the yaml file itself.
    • Changed the names of the configuration and secrets yaml files. These changes will avoid compatibility issues when using hass add-ons.

    Full Changelog: https://github.com/davidusb-geek/emhass/compare/v0.2.1...v0.2.2

    Source code(tar.gz)
    Source code(zip)
  • v0.2.1(Jan 3, 2022)

    Fixed

    • Cleaned unittest implementation

    Added

    • Added support to choose the methods for weather and load forecast from configuration file.
    • Added new load cost and power production price forecasts, mainly allowing the user to load a CSV file with their own forecast.

    Full Changelog: https://github.com/davidusb-geek/emhass/compare/v0.2.0...v0.2.1

    Source code(tar.gz)
    Source code(zip)
Owner
David
David
Quasi-static control of the centroid of quadruped robot

Quasi-static control of quadruped robot   This is a demo of the quasi-static controller for the centroid of the quadruped robot. The Quadratic Program

Junwen Cui 21 Dec 12, 2022
Raspberry Pi Pico development platform for PlatformIO

Raspberry Pi Pico development platform for PlatformIO A few words in the beginning Before experimental please Reinstall the platform Version: 1.0.0 Th

Georgi Angelov 160 Dec 23, 2022
DIY split-flap display

The goal is to make a low-cost display that's easy to fabricate at home in small/single quantities (e.g. custom materials can be ordered from Ponoko or similar, and other hardware is generally availa

Scott Bezek 2.5k Jan 05, 2023
Port of Uxn to digital hardware in the Logisim simulator

Uxn-Logisim Implements the Uxn instruction set in digital hardware. Very WIP. Contents cpu.circ - The Logisim file microcode.mc - Microcode source fil

DeltaF1 11 Mar 27, 2022
CO2Ampel - This RaspberryPi project uses weather data to estimate the share of renewable energy in the power grid

CO2Ampel This RaspberryPi project uses weather data to estimate the share of ren

Felix 4 Jan 19, 2022
NYCT-GTFS - Real-time NYC subway data parsing for humans

NYCT-GTFS - Real-time NYC subway data parsing for humans This python library provides a human-friendly, native python interface for dealing with the N

Andrew Dickinson 37 Dec 27, 2022
a weather application for the raspberry pi and the Pimorioni Inky pHAT.

raspi-weather a weather application for the raspberry pi and the Inky pHAT

Derek Caelin 59 Oct 24, 2022
3D-printable hexagonal mirror array capable of reflecting sunlight into arbitrary patterns

3D-printable hexagonal mirror array capable of reflecting sunlight into arbitrary patterns

Ben Bartlett 2.3k Dec 30, 2022
Play music on Raspberry Pi Pico Without CPU involvement

MicroPython_PIO_Music_DMA Play music on Raspberry Pi Pico Without CPU involvement This is based on PIOBeep (https://github.com/benevpi/pico_pio_buzz)

3 Nov 27, 2022
hardware design of the 250mm drone

hardware design of the 250mm drone

ZJU FAST Lab 645 Dec 25, 2022
A Home Assistant integration for Solaredge inverters

A Home Assistant integration for Solaredge inverters. Supports multiple inverters chained through RS485.

Seth 50 Dec 23, 2022
Homeautomation system created with Raspberry Pi 3 and Firebase.

Homeautomation System - Raspberry Pi 3 Desenvolvido com Python, Flask com AJAX e Firebase permite o controle local e remoto Itens necessários Raspberr

Joselino Santos 0 Mar 09, 2022
Smart EQ connect - Custom Integration for Home Assistant

Smart EQ Connect platform as a Custom Component for Home Assistant.

Rene Nulsch 2 Jan 04, 2022
Pi-hole with Inky pHAT ePaper display

Pi-hole with Inky pHAT ePaper display This is my Pi-hole with an ePaper display.

11 Sep 13, 2022
Lenovo Legion 5 Pro 2021 Linux RGB Keyboard Light Controller

Lenovo Legion 5 Pro 2021 Linux RGB Keyboard Light Controller This util allows to drive RGB keyboard light on Lenovo Legion 5 Pro 2021 Laptop Requireme

36 Dec 16, 2022
Programming of Robotics Systems course at the University of Aveiro, Portugal, 2021-2022.

Programação de Sistemas Robóticos Miguel Riem Oliveira Universidade de Aveiro 2021-2022 Projeto AtlasCar Projecto RACE IROS 2014 AtlasCar2 ATOM IROS 2

Miguel Riem de Oliveira 22 Jul 13, 2022
Ingeniamotion is a library that works over ingenialink and aims to simplify the interaction with Ingenia's drives.

Ingeniamotion Ingeniamotion is a library that works over ingenialink and aims to simplify the interaction with Ingenia's drives. Requirements Python 3

Ingenia Motion Control 7 Dec 15, 2022
Parametric open source reconstructions of Voron printed parts

The Parametric Voron This repository contains Fusion 360 reconstructions of various printed parts from the Voron printers

Matthew Lloyd 26 Dec 19, 2022
Implemented robot inverse kinematics.

robot_inverse_kinematics Project setup # put the package in the workspace $ cd ~/catkin_ws/ $ catkin_make $ source devel/setup.bash Description In thi

Jianming Han 2 Dec 08, 2022
Testing out some (stolen) DMA code for RP2040 Micropython

RP2040_micropython_dma testing out some (stolen) DMA code for RP2040 Micropython. Heavy inspiration and some code from https://iosoft.blog/2021/10/26/

2 Dec 29, 2022